Login
[x]
Log in using an account from:
Fedora Account System
Red Hat Associate
Red Hat Customer
Or login using a Red Hat Bugzilla account
Forgot Password
Login:
Hide Forgot
Create an Account
Red Hat Bugzilla – Attachment 949839 Details for
Bug 1156018
Change vm cluster show error message 'ACTION_TYPE_CPU_PROFILE_NOT_MATCH_CLUSTER'
[?]
New
Simple Search
Advanced Search
My Links
Browse
Requests
Reports
Current State
Search
Tabular reports
Graphical reports
Duplicates
Other Reports
User Changes
Plotly Reports
Bug Status
Bug Severity
Non-Defaults
|
Product Dashboard
Help
Page Help!
Bug Writing Guidelines
What's new
Browser Support Policy
5.0.4.rh83 Release notes
FAQ
Guides index
User guide
Web Services
Contact
Legal
This site requires JavaScript to be enabled to function correctly, please enable it.
engine.log
engine.log (text/plain), 1.99 MB, created by
Artyom
on 2014-10-23 12:07:32 UTC
(
hide
)
Description:
engine.log
Filename:
MIME Type:
Creator:
Artyom
Created:
2014-10-23 12:07:32 UTC
Size:
1.99 MB
patch
obsolete
>2014-10-15 03:28:28,035 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-94) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 418 seconds while maximum configured value is 300 seconds. >2014-10-15 03:35:35,001 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-5) Start deleteAgedOutAuditLogs >2014-10-15 03:35:35,001 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-93) Start deleteAgedOutCommandEntities >2014-10-15 03:35:35,011 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-93) Finished deleteAgedOutCommandEntities >2014-10-15 03:35:35,032 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-5) Finished deleteAgedOutAuditLogs >2014-10-15 03:59:28,525 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-18) Attempting to update VMs/Templates Ovf. >2014-10-15 04:00:42,609 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-96) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-15 04:28:32,078 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-10) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-15 04:59:28,558 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-64) Attempting to update VMs/Templates Ovf. >2014-10-15 05:00:48,041 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-52) [7b116c2f] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-15 05:28:36,374 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-75) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-15 05:59:28,575 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-14) Attempting to update VMs/Templates Ovf. >2014-10-15 06:00:54,012 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-46) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 418 seconds while maximum configured value is 300 seconds. >2014-10-15 06:28:41,632 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-15 06:59:28,590 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-3) Attempting to update VMs/Templates Ovf. >2014-10-15 07:01:00,786 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-61) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-15 07:28:46,725 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-41) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-15 07:59:28,607 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-71) Attempting to update VMs/Templates Ovf. >2014-10-15 08:01:05,642 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-15 08:28:50,304 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-89) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-15 08:59:28,623 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-13) Attempting to update VMs/Templates Ovf. >2014-10-15 09:01:10,322 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-93) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-15 09:28:53,981 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-78) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-15 09:59:28,642 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-35) Attempting to update VMs/Templates Ovf. >2014-10-15 10:01:15,085 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-61) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-15 10:28:58,496 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-81) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-15 10:59:28,658 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-38) Attempting to update VMs/Templates Ovf. >2014-10-15 11:01:20,089 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-74) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-15 11:05:38,292 INFO [org.ovirt.engine.ui.frontend.server.gwt.plugin.PluginDataManager] (ajp-/127.0.0.1:8702-4) Reading UI plugin descriptor [/usr/share/ovirt-engine/ui-plugins/redhat_support_plugin_rhev.json] >2014-10-15 11:05:38,293 INFO [org.ovirt.engine.ui.frontend.server.gwt.plugin.PluginDataManager] (ajp-/127.0.0.1:8702-4) Reading UI plugin configuration [/etc/ovirt-engine/ui-plugins/redhat_support_plugin_rhev-config.json] >2014-10-15 11:05:38,324 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-4) Invalid locale found in configuration: >2014-10-15 11:05:38,324 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-4) Invalid locale found in configuration: >2014-10-15 11:29:03,077 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-75) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-15 11:59:28,673 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-36) Attempting to update VMs/Templates Ovf. >2014-10-15 12:01:25,505 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-8) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-15 12:29:08,014 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-98) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 418 seconds while maximum configured value is 300 seconds. >2014-10-15 12:59:28,694 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-95) Attempting to update VMs/Templates Ovf. >2014-10-15 13:01:31,630 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-34) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-15 13:29:13,257 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-90) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-15 13:59:28,711 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-47) Attempting to update VMs/Templates Ovf. >2014-10-15 14:01:37,616 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-67) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-15 14:29:17,454 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-72) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-15 14:59:28,728 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-92) Attempting to update VMs/Templates Ovf. >2014-10-15 15:01:44,744 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-64) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-15 15:29:25,024 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-53) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 418 seconds while maximum configured value is 300 seconds. >2014-10-15 15:59:28,748 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-9) Attempting to update VMs/Templates Ovf. >2014-10-15 16:01:50,995 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-91) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-15 16:29:29,080 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-48) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-15 16:59:28,764 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-28) Attempting to update VMs/Templates Ovf. >2014-10-15 17:01:56,907 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-10) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-15 17:29:33,764 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-89) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-15 17:59:28,777 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-68) Attempting to update VMs/Templates Ovf. >2014-10-15 18:02:05,186 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-48) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-15 18:29:38,358 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-93) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-15 18:59:28,791 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-95) Attempting to update VMs/Templates Ovf. >2014-10-15 19:02:11,039 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-71) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 418 seconds while maximum configured value is 300 seconds. >2014-10-15 19:29:43,197 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-29) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-15 19:59:28,806 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-3) Attempting to update VMs/Templates Ovf. >2014-10-15 20:02:18,660 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-76) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-15 20:29:47,714 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-88) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-15 20:59:28,822 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-33) Attempting to update VMs/Templates Ovf. >2014-10-15 21:02:24,128 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-14) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-15 21:29:51,299 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-94) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-15 21:59:28,838 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-59) Attempting to update VMs/Templates Ovf. >2014-10-15 22:02:29,679 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-45) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-15 22:29:54,980 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-29) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-15 22:59:28,883 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-55) Attempting to update VMs/Templates Ovf. >2014-10-15 23:02:35,753 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-38) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-15 23:29:58,529 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-26) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-15 23:59:28,897 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-26) Attempting to update VMs/Templates Ovf. >2014-10-16 00:02:41,637 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-67) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-16 00:30:01,931 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-5) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-16 00:59:28,912 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-79) Attempting to update VMs/Templates Ovf. >2014-10-16 01:02:46,896 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-81) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-16 01:30:05,638 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-44) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-16 01:59:28,927 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-53) Attempting to update VMs/Templates Ovf. >2014-10-16 02:02:52,858 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-24) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-16 02:30:09,539 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-16 02:59:28,943 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-43) Attempting to update VMs/Templates Ovf. >2014-10-16 03:02:58,618 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-64) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-16 03:30:14,109 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-53) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-16 03:35:35,001 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-25) Start deleteAgedOutAuditLogs >2014-10-16 03:35:35,001 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-64) Start deleteAgedOutCommandEntities >2014-10-16 03:35:35,004 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-64) Finished deleteAgedOutCommandEntities >2014-10-16 03:35:35,012 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-25) Finished deleteAgedOutAuditLogs >2014-10-16 03:59:28,957 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-51) Attempting to update VMs/Templates Ovf. >2014-10-16 04:03:04,045 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-26) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-16 04:30:17,688 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-39) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-16 04:59:28,972 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-8) Attempting to update VMs/Templates Ovf. >2014-10-16 05:03:09,144 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-97) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-16 05:30:22,361 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-23) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-16 05:59:28,994 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-23) Attempting to update VMs/Templates Ovf. >2014-10-16 06:03:14,690 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-82) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-16 06:30:26,999 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-95) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-16 06:59:29,013 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-3) Attempting to update VMs/Templates Ovf. >2014-10-16 07:03:19,575 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-24) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-16 07:30:30,655 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-54) [3a38f005] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-16 07:59:29,034 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-45) Attempting to update VMs/Templates Ovf. >2014-10-16 08:03:24,712 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-50) [4314b1db] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-16 08:30:34,354 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-72) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-16 08:59:29,054 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-19) Attempting to update VMs/Templates Ovf. >2014-10-16 09:03:30,058 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-16 09:30:38,274 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-50) [4314b1db] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-16 09:59:29,073 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-56) Attempting to update VMs/Templates Ovf. >2014-10-16 10:03:35,133 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-66) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-16 10:30:42,961 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-56) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-16 10:59:29,089 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-49) Attempting to update VMs/Templates Ovf. >2014-10-16 11:03:40,852 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-18) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-16 11:30:47,596 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-60) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-16 11:59:29,107 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-11) [43a5b796] Attempting to update VMs/Templates Ovf. >2014-10-16 12:03:47,219 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-92) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-16 12:30:51,869 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-67) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-16 12:59:29,128 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-70) Attempting to update VMs/Templates Ovf. >2014-10-16 13:03:53,030 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-1) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 418 seconds while maximum configured value is 300 seconds. >2014-10-16 13:30:57,049 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-61) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-16 13:59:29,143 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-82) Attempting to update VMs/Templates Ovf. >2014-10-16 14:03:58,172 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-74) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-16 14:31:01,811 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-52) [7b116c2f] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-16 14:59:29,158 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-5) Attempting to update VMs/Templates Ovf. >2014-10-16 15:04:04,410 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-49) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-16 15:31:06,245 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-82) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-16 15:59:29,173 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-68) Attempting to update VMs/Templates Ovf. >2014-10-16 16:04:09,869 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-82) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-16 16:31:10,935 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-32) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-16 16:59:29,189 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-100) Attempting to update VMs/Templates Ovf. >2014-10-16 17:04:15,370 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-30) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-16 17:31:15,186 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-16 17:59:29,204 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-36) Attempting to update VMs/Templates Ovf. >2014-10-16 18:04:23,093 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-67) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-16 18:31:20,900 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-94) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-16 18:59:29,219 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-84) Attempting to update VMs/Templates Ovf. >2014-10-16 19:04:28,470 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-26) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-16 19:31:25,322 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-54) [3a38f005] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-16 19:59:29,234 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-76) Attempting to update VMs/Templates Ovf. >2014-10-16 20:04:33,760 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-89) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-16 20:31:29,452 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-16 20:59:29,249 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-33) Attempting to update VMs/Templates Ovf. >2014-10-16 21:04:38,659 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-18) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-16 21:31:33,880 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-70) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-16 21:59:29,264 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-91) Attempting to update VMs/Templates Ovf. >2014-10-16 22:04:43,960 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-28) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-16 22:31:38,017 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-81) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 418 seconds while maximum configured value is 300 seconds. >2014-10-16 22:59:29,278 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-37) Attempting to update VMs/Templates Ovf. >2014-10-16 23:04:49,239 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-3) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-16 23:31:42,326 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-55) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-16 23:59:29,293 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-20) Attempting to update VMs/Templates Ovf. >2014-10-17 00:04:54,355 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-93) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-17 00:31:46,767 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-94) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-17 00:59:29,307 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-18) Attempting to update VMs/Templates Ovf. >2014-10-17 01:04:59,683 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-54) [3a38f005] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-17 01:31:51,236 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-41) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-17 01:59:29,322 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-70) Attempting to update VMs/Templates Ovf. >2014-10-17 02:05:05,909 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-17 02:31:56,980 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-80) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-17 02:59:29,337 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-47) Attempting to update VMs/Templates Ovf. >2014-10-17 03:05:12,821 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-86) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-17 03:32:01,745 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-78) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-17 03:35:35,001 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-79) Start deleteAgedOutAuditLogs >2014-10-17 03:35:35,002 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-21) Start deleteAgedOutCommandEntities >2014-10-17 03:35:35,005 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-21) Finished deleteAgedOutCommandEntities >2014-10-17 03:35:35,012 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-79) Finished deleteAgedOutAuditLogs >2014-10-17 03:59:29,349 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-57) Attempting to update VMs/Templates Ovf. >2014-10-17 04:05:18,365 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-99) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-17 04:32:06,350 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) [43a5b796] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-17 04:59:29,369 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-10) Attempting to update VMs/Templates Ovf. >2014-10-17 05:05:23,950 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-50) [4314b1db] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-17 05:32:11,325 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-17 05:59:29,382 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-67) Attempting to update VMs/Templates Ovf. >2014-10-17 06:05:29,557 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-46) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-17 06:32:16,356 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-84) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-17 06:59:29,398 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-50) [4314b1db] Attempting to update VMs/Templates Ovf. >2014-10-17 07:05:36,218 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-97) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-17 07:32:21,132 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-65) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-17 07:59:29,412 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-76) Attempting to update VMs/Templates Ovf. >2014-10-17 08:05:42,852 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-79) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-17 08:32:25,310 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-97) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-17 08:59:29,426 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-73) Attempting to update VMs/Templates Ovf. >2014-10-17 09:05:49,400 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-14) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-17 09:32:29,221 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-33) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-17 09:59:29,441 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-81) Attempting to update VMs/Templates Ovf. >2014-10-17 10:05:53,559 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-35) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-17 10:32:33,601 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-65) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-17 10:59:29,455 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-89) Attempting to update VMs/Templates Ovf. >2014-10-17 11:05:57,869 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-92) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-17 11:32:37,930 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-27) [47301dc8] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-17 11:59:29,470 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-88) Attempting to update VMs/Templates Ovf. >2014-10-17 12:06:03,094 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-67) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-17 12:32:42,121 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-18) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-17 12:59:29,484 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-75) Attempting to update VMs/Templates Ovf. >2014-10-17 13:06:08,496 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-93) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-17 13:32:46,650 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-22) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-17 13:59:29,500 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-33) Attempting to update VMs/Templates Ovf. >2014-10-17 14:06:13,558 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-19) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-17 14:32:50,901 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-43) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-17 14:59:29,515 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-41) Attempting to update VMs/Templates Ovf. >2014-10-17 15:06:18,357 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-59) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-17 15:32:55,074 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-10) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-17 15:59:29,528 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-40) Attempting to update VMs/Templates Ovf. >2014-10-17 16:06:24,030 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-83) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 418 seconds while maximum configured value is 300 seconds. >2014-10-17 16:32:59,503 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-81) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-17 16:59:29,542 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-79) Attempting to update VMs/Templates Ovf. >2014-10-17 17:06:30,864 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-24) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-17 17:33:03,774 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-2) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-17 17:59:29,556 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-66) Attempting to update VMs/Templates Ovf. >2014-10-17 18:06:35,706 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-21) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-17 18:33:07,835 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-100) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-17 18:59:29,570 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-44) Attempting to update VMs/Templates Ovf. >2014-10-17 19:06:41,322 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-64) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-17 19:33:12,067 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-36) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-17 19:59:29,584 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-33) Attempting to update VMs/Templates Ovf. >2014-10-17 20:06:46,993 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-52) [7b116c2f] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-17 20:33:16,491 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-99) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-17 20:59:29,598 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-39) Attempting to update VMs/Templates Ovf. >2014-10-17 21:06:51,871 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-5) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-17 21:33:20,900 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-42) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-17 21:59:29,612 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-22) Attempting to update VMs/Templates Ovf. >2014-10-17 22:06:57,359 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-22) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-17 22:33:24,957 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-82) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-17 22:59:29,626 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-67) Attempting to update VMs/Templates Ovf. >2014-10-17 23:07:01,661 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-61) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-17 23:33:29,220 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-61) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-17 23:59:29,644 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-53) Attempting to update VMs/Templates Ovf. >2014-10-18 00:07:06,877 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-57) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-18 00:33:33,888 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-18 00:59:29,657 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-33) Attempting to update VMs/Templates Ovf. >2014-10-18 01:07:11,898 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-40) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-18 01:33:39,797 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-19) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-18 01:59:29,672 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-72) Attempting to update VMs/Templates Ovf. >2014-10-18 02:07:16,955 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-52) [7b116c2f] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-18 02:33:43,884 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-94) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-18 02:59:29,687 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-30) Attempting to update VMs/Templates Ovf. >2014-10-18 03:07:21,779 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-73) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-18 03:33:48,189 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-41) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-18 03:35:35,001 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-48) Start deleteAgedOutAuditLogs >2014-10-18 03:35:35,001 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-98) Start deleteAgedOutCommandEntities >2014-10-18 03:35:35,004 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-98) Finished deleteAgedOutCommandEntities >2014-10-18 03:35:35,023 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-48) Finished deleteAgedOutAuditLogs >2014-10-18 03:59:29,712 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-62) Attempting to update VMs/Templates Ovf. >2014-10-18 04:07:27,340 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-88) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-18 04:33:52,541 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-68) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-18 04:59:29,747 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-55) Attempting to update VMs/Templates Ovf. >2014-10-18 05:07:33,266 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-47) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-18 05:33:56,628 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-50) [4314b1db] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-18 05:59:29,773 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-31) [31332b57] Attempting to update VMs/Templates Ovf. >2014-10-18 06:07:38,589 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-17) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-18 06:34:02,295 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-98) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-18 06:59:29,788 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-12) Attempting to update VMs/Templates Ovf. >2014-10-18 07:07:44,091 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-81) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-18 07:34:06,596 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-2) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-18 07:59:29,802 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-72) Attempting to update VMs/Templates Ovf. >2014-10-18 08:07:49,977 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-74) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-18 08:34:10,925 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-80) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-18 08:59:29,816 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-14) Attempting to update VMs/Templates Ovf. >2014-10-18 09:07:54,428 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-16) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-18 09:34:15,287 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-28) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-18 09:59:29,830 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-62) Attempting to update VMs/Templates Ovf. >2014-10-18 10:08:00,792 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-44) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-18 10:34:19,809 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-62) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-18 10:59:29,844 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-38) Attempting to update VMs/Templates Ovf. >2014-10-18 11:08:06,455 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-18) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-18 11:34:24,024 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-97) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 418 seconds while maximum configured value is 300 seconds. >2014-10-18 11:59:29,860 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-4) Attempting to update VMs/Templates Ovf. >2014-10-18 12:08:11,659 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-17) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-18 12:34:28,770 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-69) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-18 12:59:29,874 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-10) Attempting to update VMs/Templates Ovf. >2014-10-18 13:08:16,565 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-12) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-18 13:34:33,209 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-65) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-18 13:59:29,886 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-29) Attempting to update VMs/Templates Ovf. >2014-10-18 14:08:21,989 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-54) [3a38f005] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-18 14:34:37,418 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-89) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-18 14:59:29,899 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-67) Attempting to update VMs/Templates Ovf. >2014-10-18 15:08:27,503 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-31) [31332b57] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-18 15:34:41,816 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-27) [47301dc8] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-18 15:59:29,912 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-43) Attempting to update VMs/Templates Ovf. >2014-10-18 16:08:34,027 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-28) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 418 seconds while maximum configured value is 300 seconds. >2014-10-18 16:34:46,114 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-43) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-18 16:59:29,957 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-78) Attempting to update VMs/Templates Ovf. >2014-10-18 17:08:40,351 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-81) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-18 17:34:49,869 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-48) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-18 17:59:29,973 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-70) Attempting to update VMs/Templates Ovf. >2014-10-18 18:08:45,972 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-9) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-18 18:34:54,208 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-16) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-18 18:59:29,986 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-5) Attempting to update VMs/Templates Ovf. >2014-10-18 19:08:51,774 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-68) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-18 19:34:58,499 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-18 19:59:30,000 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-18) Attempting to update VMs/Templates Ovf. >2014-10-18 20:08:58,637 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-53) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-18 20:35:02,596 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-12) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-18 20:59:30,013 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-68) Attempting to update VMs/Templates Ovf. >2014-10-18 21:09:04,296 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) [43a5b796] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-18 21:35:06,465 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-57) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-18 21:59:30,028 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-1) Attempting to update VMs/Templates Ovf. >2014-10-18 22:09:09,180 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-63) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-18 22:35:10,496 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-78) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-18 22:59:30,042 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-94) Attempting to update VMs/Templates Ovf. >2014-10-18 23:09:14,217 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-58) [53ccdc22] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-18 23:35:14,443 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-98) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-18 23:59:30,057 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-89) Attempting to update VMs/Templates Ovf. >2014-10-19 00:09:19,480 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-57) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-19 00:35:18,868 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-35) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-19 00:59:30,072 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-23) Attempting to update VMs/Templates Ovf. >2014-10-19 01:09:25,452 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-8) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-19 01:35:22,751 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-92) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-19 01:59:30,088 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-26) Attempting to update VMs/Templates Ovf. >2014-10-19 02:09:31,303 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-52) [7b116c2f] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-19 02:35:27,278 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-2) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-19 02:59:30,103 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-79) Attempting to update VMs/Templates Ovf. >2014-10-19 03:09:36,793 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-16) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-19 03:35:32,086 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-28) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-19 03:35:35,000 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-67) Start deleteAgedOutAuditLogs >2014-10-19 03:35:35,001 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-10) Start deleteAgedOutCommandEntities >2014-10-19 03:35:35,004 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-10) Finished deleteAgedOutCommandEntities >2014-10-19 03:35:35,011 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-67) Finished deleteAgedOutAuditLogs >2014-10-19 03:59:30,117 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-49) Attempting to update VMs/Templates Ovf. >2014-10-19 04:09:42,768 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-71) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-19 04:35:36,003 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-92) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-19 04:59:30,138 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-34) Attempting to update VMs/Templates Ovf. >2014-10-19 05:09:49,017 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-21) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 418 seconds while maximum configured value is 300 seconds. >2014-10-19 05:35:40,324 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-50) [4314b1db] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-19 05:59:30,152 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-85) Attempting to update VMs/Templates Ovf. >2014-10-19 06:09:55,457 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-88) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-19 06:35:44,363 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-68) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-19 06:59:30,165 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-10) Attempting to update VMs/Templates Ovf. >2014-10-19 07:10:00,277 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-24) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-19 07:35:48,562 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-94) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-19 07:59:30,179 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-77) Attempting to update VMs/Templates Ovf. >2014-10-19 08:10:05,667 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-84) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-19 08:35:53,377 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) [43a5b796] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-19 08:59:30,207 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-29) Attempting to update VMs/Templates Ovf. >2014-10-19 09:10:12,470 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-70) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-19 09:36:02,095 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-26) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-19 09:59:30,222 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-58) [53ccdc22] Attempting to update VMs/Templates Ovf. >2014-10-19 10:10:20,597 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-39) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-19 10:36:07,749 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-88) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-19 10:59:30,236 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-33) Attempting to update VMs/Templates Ovf. >2014-10-19 11:10:25,651 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-78) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-19 11:36:11,819 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-13) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-19 11:59:30,252 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-71) Attempting to update VMs/Templates Ovf. >2014-10-19 12:10:31,131 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-13) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-19 12:36:15,736 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-19 12:42:22,480 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ListVDSCommand] (DefaultQuartzScheduler_Worker-93) Command ListVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-19 12:42:22,481 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-93) Host alma06.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 80 seconds and after that an attempt to fence the host will be issued. >2014-10-19 12:42:22,488 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-93) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 80 seconds and after that an attempt to fence the host will be issued. >2014-10-19 12:42:22,489 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-93) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.ListVDSCommand.executeVdsBrokerCommand(ListVDSCommand.java:24) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:418) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.fetchRunningVms(VdsUpdateRunTimeInfo.java:1000) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVmStats(VdsUpdateRunTimeInfo.java:949) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:667) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-19 12:42:22,496 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-93) Failed to refresh VDS , vds = 1c39bd88-8672-408f-a593-60d6ef8f647c : alma06.qa.lab.tlv.redhat.com, VDS Network Error, continuing. >VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-19 12:42:24,985 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-27) [47301dc8] Command SpmStatusVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-19 12:42:24,988 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-27) [55c797c1] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: f8f5eaee-8fd0-4b45-87db-62d61b03a916 Type: StoragePool >2014-10-19 12:42:24,991 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-27) [55c797c1] Storage Pool f8f5eaee-8fd0-4b45-87db-62d61b03a916 - Updating Storage Domain a1341057-9cf8-4750-a734-656d05a5d2c5 status from Active to Unknown, reason : null >2014-10-19 12:42:25,002 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-27) [55c797c1] Correlation ID: 55c797c1, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center dc_35. Setting Data Center status to Non Responsive (On host alma06.qa.lab.tlv.redhat.com, Error: Network error during communication with the Host.). >2014-10-19 12:42:25,146 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-19 12:42:27,147 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:42:27,148 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:42:27,148 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-19 12:42:27,148 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-27) Exception during connection: java.util.concurrent.ExecutionException: java.rmi.ConnectException: Connection timeout > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:122) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcIIrsServer.getStoragePoolInfo(JsonRpcIIrsServer.java:381) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.GetStoragePoolInfoVDSCommand.executeIrsBrokerCommand(GetStoragePoolInfoVDSCommand.java:31) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand.executeVDSCommand(IrsBrokerCommand.java:156) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:418) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData.proceedStoragePoolStats(IrsProxyData.java:293) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData._updatingTimer_Elapsed(IrsProxyData.java:187) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor92.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:42:27,151 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-27) [55c797c1] IrsBroker::Failed::GetStoragePoolInfoVDS >2014-10-19 12:42:27,152 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-27) [55c797c1] ERROR, org.ovirt.engine.core.vdsbroker.irsbroker.GetStoragePoolInfoVDSCommand, exception: XmlRpcRunTimeException: Connection issues during send request, log id: 3f503a2: org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcRunTimeException: Connection issues during send request > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:62) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcIIrsServer.getStoragePoolInfo(JsonRpcIIrsServer.java:381) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.GetStoragePoolInfoVDSCommand.executeIrsBrokerCommand(GetStoragePoolInfoVDSCommand.java:31) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand.executeVDSCommand(IrsBrokerCommand.java:156) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:418) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData.proceedStoragePoolStats(IrsProxyData.java:293) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData._updatingTimer_Elapsed(IrsProxyData.java:187) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor92.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: java.util.concurrent.ExecutionException: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:130) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > ... 14 more >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:42:27,153 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.GetStoragePoolInfoVDSCommand] (DefaultQuartzScheduler_Worker-27) [55c797c1] Command GetStoragePoolInfoVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = true) execution failed. Exception: IRSProtocolException: IRSProtocolException: >2014-10-19 12:42:29,148 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:42:29,148 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:42:29,148 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-34) Exception during connection: java.util.concurrent.ExecutionException: java.rmi.ConnectException: Connection timeout > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:122) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:42:29,152 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-34) Command GetCapabilitiesVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c]) execution failed. Exception: VDSNetworkException: java.rmi.ConnectException: Connection timeout >2014-10-19 12:42:29,152 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-34) Host alma06.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 80 seconds and after that an attempt to fence the host will be issued. >2014-10-19 12:42:29,157 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-34) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.rmi.ConnectException: Connection timeout > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:42:29,161 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-34) Failed to refresh VDS , vds = 1c39bd88-8672-408f-a593-60d6ef8f647c : alma06.qa.lab.tlv.redhat.com, VDS Network Error, continuing. >java.rmi.ConnectException: Connection timeout >2014-10-19 12:42:32,167 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-19 12:42:34,168 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:42:34,168 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:42:34,169 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-65) Exception during connection: java.util.concurrent.ExecutionException: java.rmi.ConnectException: Connection timeout > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:122) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:42:34,172 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-65) Command GetCapabilitiesVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c]) execution failed. Exception: VDSNetworkException: java.rmi.ConnectException: Connection timeout >2014-10-19 12:42:34,172 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-65) Host alma06.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 80 seconds and after that an attempt to fence the host will be issued. >2014-10-19 12:42:34,244 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (DefaultQuartzScheduler_Worker-65) START, SetVmStatusVDSCommand( vmId = 686605cb-13f6-41b7-bd14-04ae79b55799, status = Unknown, exit status = Normal), log id: 67b1c132 >2014-10-19 12:42:34,249 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (DefaultQuartzScheduler_Worker-65) FINISH, SetVmStatusVDSCommand, log id: 67b1c132 >2014-10-19 12:42:34,325 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-65) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM iscsi_vm was set to the Unknown status. >2014-10-19 12:42:34,325 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-65) Server failed to respond, vds_id = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds_name = alma06.qa.lab.tlv.redhat.com, vm_count = 1, spm_status = SPM, non-responsive_timeout (seconds) = 80, error = java.rmi.ConnectException: Connection timeout >2014-10-19 12:42:34,332 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-65) Correlation ID: null, Call Stack: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.rmi.ConnectException: Connection timeout > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) >, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com is non responsive. >2014-10-19 12:42:34,334 INFO [org.ovirt.engine.core.bll.VdsEventListener] (org.ovirt.thread.pool-7-thread-30) ResourceManager::vdsNotResponding entered for Host 1c39bd88-8672-408f-a593-60d6ef8f647c, alma06.qa.lab.tlv.redhat.com >2014-10-19 12:42:34,339 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-65) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.rmi.ConnectException: Connection timeout > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:42:34,358 INFO [org.ovirt.engine.core.bll.SshSoftFencingCommand] (org.ovirt.thread.pool-7-thread-30) [787e1d51] Running command: SshSoftFencingCommand internal: true. Entities affected : ID: 1c39bd88-8672-408f-a593-60d6ef8f647c Type: VDS >2014-10-19 12:42:34,361 INFO [org.ovirt.engine.core.bll.SshSoftFencingCommand] (org.ovirt.thread.pool-7-thread-30) [787e1d51] Opening SSH Soft Fencing session on host alma06.qa.lab.tlv.redhat.com >2014-10-19 12:42:37,157 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-19 12:42:39,158 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:42:39,158 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:42:39,159 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-19 12:42:39,158 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-37) Exception during connection: java.util.concurrent.ExecutionException: java.rmi.ConnectException: Connection timeout > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:122) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.spmStatus(JsonRpcVdsServer.java:794) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand.executeVdsBrokerCommand(SpmStatusVDSCommand.java:24) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:418) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData.proceedStoragePoolStats(IrsProxyData.java:221) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData._updatingTimer_Elapsed(IrsProxyData.java:187) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor92.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:42:39,163 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-37) Command SpmStatusVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916) execution failed. Exception: VDSNetworkException: java.rmi.ConnectException: Connection timeout >2014-10-19 12:42:41,159 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:42:41,159 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:42:41,159 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-19 12:42:41,159 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-54) Exception during connection: java.util.concurrent.ExecutionException: java.rmi.ConnectException: Connection timeout > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:122) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:42:41,162 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-54) [3a38f005] Command GetCapabilitiesVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c]) execution failed. Exception: VDSNetworkException: java.rmi.ConnectException: Connection timeout >2014-10-19 12:42:41,163 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-54) [3a38f005] Host alma06.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-10-19 12:42:41,171 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-54) [3a38f005] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.rmi.ConnectException: Connection timeout > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:42:43,160 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:42:43,160 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:42:43,160 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-37) Exception during connection: java.util.concurrent.ExecutionException: java.rmi.ConnectException: Connection timeout > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:122) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcIIrsServer.getStoragePoolInfo(JsonRpcIIrsServer.java:381) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.GetStoragePoolInfoVDSCommand.executeIrsBrokerCommand(GetStoragePoolInfoVDSCommand.java:31) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand.executeVDSCommand(IrsBrokerCommand.java:156) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:418) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData.proceedStoragePoolStats(IrsProxyData.java:293) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData._updatingTimer_Elapsed(IrsProxyData.java:187) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor92.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:42:43,163 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-37) IrsBroker::Failed::GetStoragePoolInfoVDS >2014-10-19 12:42:43,164 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-37) ERROR, org.ovirt.engine.core.vdsbroker.irsbroker.GetStoragePoolInfoVDSCommand, exception: XmlRpcRunTimeException: Connection issues during send request, log id: 4bd329b4: org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcRunTimeException: Connection issues during send request > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:62) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcIIrsServer.getStoragePoolInfo(JsonRpcIIrsServer.java:381) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.GetStoragePoolInfoVDSCommand.executeIrsBrokerCommand(GetStoragePoolInfoVDSCommand.java:31) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand.executeVDSCommand(IrsBrokerCommand.java:156) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:418) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData.proceedStoragePoolStats(IrsProxyData.java:293) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData._updatingTimer_Elapsed(IrsProxyData.java:187) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor92.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: java.util.concurrent.ExecutionException: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:130) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > ... 14 more >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:42:43,166 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.GetStoragePoolInfoVDSCommand] (DefaultQuartzScheduler_Worker-37) Command GetStoragePoolInfoVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = true) execution failed. Exception: IRSProtocolException: IRSProtocolException: >2014-10-19 12:42:44,181 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-19 12:42:46,181 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:42:46,181 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:42:46,181 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-35) Exception during connection: java.util.concurrent.ExecutionException: java.rmi.ConnectException: Connection timeout > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:122) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:42:46,184 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-35) Command GetCapabilitiesVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c]) execution failed. Exception: VDSNetworkException: java.rmi.ConnectException: Connection timeout >2014-10-19 12:42:46,184 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-35) Host alma06.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-10-19 12:42:46,193 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-35) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.rmi.ConnectException: Connection timeout > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:42:49,203 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-19 12:42:51,204 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:42:51,204 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:42:51,205 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-58) Exception during connection: java.util.concurrent.ExecutionException: java.rmi.ConnectException: Connection timeout > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:122) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:42:51,208 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-58) [53ccdc22] Command GetCapabilitiesVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c]) execution failed. Exception: VDSNetworkException: java.rmi.ConnectException: Connection timeout >2014-10-19 12:42:51,208 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-58) [53ccdc22] Host alma06.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-10-19 12:42:51,217 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-58) [53ccdc22] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.rmi.ConnectException: Connection timeout > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:42:53,172 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-19 12:42:55,172 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:42:55,172 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:42:55,173 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-19 12:42:55,172 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-12) Exception during connection: java.util.concurrent.ExecutionException: java.rmi.ConnectException: Connection timeout > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:122) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.spmStatus(JsonRpcVdsServer.java:794) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand.executeVdsBrokerCommand(SpmStatusVDSCommand.java:24) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:418) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData.proceedStoragePoolStats(IrsProxyData.java:221) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData._updatingTimer_Elapsed(IrsProxyData.java:187) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor92.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:42:55,177 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-12) Command SpmStatusVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916) execution failed. Exception: VDSNetworkException: java.rmi.ConnectException: Connection timeout >2014-10-19 12:42:57,174 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:42:57,174 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:42:57,174 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-19 12:42:57,174 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-100) Exception during connection: java.util.concurrent.ExecutionException: java.rmi.ConnectException: Connection timeout > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:122) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:42:57,177 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-100) Command GetCapabilitiesVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c]) execution failed. Exception: VDSNetworkException: java.rmi.ConnectException: Connection timeout >2014-10-19 12:42:57,177 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-100) Host alma06.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-10-19 12:42:57,185 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-100) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.rmi.ConnectException: Connection timeout > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:42:59,174 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:42:59,174 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:42:59,174 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-12) Exception during connection: java.util.concurrent.ExecutionException: java.rmi.ConnectException: Connection timeout > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:122) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcIIrsServer.getStoragePoolInfo(JsonRpcIIrsServer.java:381) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.GetStoragePoolInfoVDSCommand.executeIrsBrokerCommand(GetStoragePoolInfoVDSCommand.java:31) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand.executeVDSCommand(IrsBrokerCommand.java:156) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:418) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData.proceedStoragePoolStats(IrsProxyData.java:293) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData._updatingTimer_Elapsed(IrsProxyData.java:187) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor92.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:42:59,177 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-12) IrsBroker::Failed::GetStoragePoolInfoVDS >2014-10-19 12:42:59,177 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-12) ERROR, org.ovirt.engine.core.vdsbroker.irsbroker.GetStoragePoolInfoVDSCommand, exception: XmlRpcRunTimeException: Connection issues during send request, log id: 36b61d93: org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcRunTimeException: Connection issues during send request > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:62) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcIIrsServer.getStoragePoolInfo(JsonRpcIIrsServer.java:381) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.GetStoragePoolInfoVDSCommand.executeIrsBrokerCommand(GetStoragePoolInfoVDSCommand.java:31) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand.executeVDSCommand(IrsBrokerCommand.java:156) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:418) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData.proceedStoragePoolStats(IrsProxyData.java:293) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData._updatingTimer_Elapsed(IrsProxyData.java:187) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor92.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: java.util.concurrent.ExecutionException: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:130) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > ... 14 more >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:42:59,178 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.GetStoragePoolInfoVDSCommand] (DefaultQuartzScheduler_Worker-12) Command GetStoragePoolInfoVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = true) execution failed. Exception: IRSProtocolException: IRSProtocolException: >2014-10-19 12:43:00,193 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-19 12:43:02,194 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:02,194 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:02,194 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-5) Exception during connection: java.util.concurrent.ExecutionException: java.rmi.ConnectException: Connection timeout > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:122) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:02,197 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-5) Command GetCapabilitiesVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c]) execution failed. Exception: VDSNetworkException: java.rmi.ConnectException: Connection timeout >2014-10-19 12:43:02,197 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-5) Host alma06.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-10-19 12:43:02,205 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-5) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.rmi.ConnectException: Connection timeout > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:05,215 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-19 12:43:07,215 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:07,215 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:07,216 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-89) Exception during connection: java.util.concurrent.ExecutionException: java.rmi.ConnectException: Connection timeout > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:122) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:07,219 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-89) Command GetCapabilitiesVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c]) execution failed. Exception: VDSNetworkException: java.rmi.ConnectException: Connection timeout >2014-10-19 12:43:07,219 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-89) Host alma06.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-10-19 12:43:07,228 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-89) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.rmi.ConnectException: Connection timeout > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:09,183 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-19 12:43:11,183 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:11,183 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:11,184 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-19 12:43:11,183 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-95) Exception during connection: java.util.concurrent.ExecutionException: java.rmi.ConnectException: Connection timeout > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:122) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.spmStatus(JsonRpcVdsServer.java:794) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand.executeVdsBrokerCommand(SpmStatusVDSCommand.java:24) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:418) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData.proceedStoragePoolStats(IrsProxyData.java:221) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData._updatingTimer_Elapsed(IrsProxyData.java:187) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor92.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:11,187 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-95) Command SpmStatusVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916) execution failed. Exception: VDSNetworkException: java.rmi.ConnectException: Connection timeout >2014-10-19 12:43:13,184 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:13,184 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:13,185 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-19 12:43:13,185 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-54) Exception during connection: java.util.concurrent.ExecutionException: java.rmi.ConnectException: Connection timeout > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:122) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:13,188 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-54) [3a38f005] Command GetCapabilitiesVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c]) execution failed. Exception: VDSNetworkException: java.rmi.ConnectException: Connection timeout >2014-10-19 12:43:13,188 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-54) [3a38f005] Host alma06.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-10-19 12:43:13,196 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-54) [3a38f005] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.rmi.ConnectException: Connection timeout > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:15,185 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:15,185 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:15,185 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-95) Exception during connection: java.util.concurrent.ExecutionException: java.rmi.ConnectException: Connection timeout > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:122) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcIIrsServer.getStoragePoolInfo(JsonRpcIIrsServer.java:381) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.GetStoragePoolInfoVDSCommand.executeIrsBrokerCommand(GetStoragePoolInfoVDSCommand.java:31) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand.executeVDSCommand(IrsBrokerCommand.java:156) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:418) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData.proceedStoragePoolStats(IrsProxyData.java:293) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData._updatingTimer_Elapsed(IrsProxyData.java:187) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor92.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:15,188 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-95) IrsBroker::Failed::GetStoragePoolInfoVDS >2014-10-19 12:43:15,188 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-95) ERROR, org.ovirt.engine.core.vdsbroker.irsbroker.GetStoragePoolInfoVDSCommand, exception: XmlRpcRunTimeException: Connection issues during send request, log id: 52005d0: org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcRunTimeException: Connection issues during send request > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:62) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcIIrsServer.getStoragePoolInfo(JsonRpcIIrsServer.java:381) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.GetStoragePoolInfoVDSCommand.executeIrsBrokerCommand(GetStoragePoolInfoVDSCommand.java:31) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand.executeVDSCommand(IrsBrokerCommand.java:156) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:418) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData.proceedStoragePoolStats(IrsProxyData.java:293) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData._updatingTimer_Elapsed(IrsProxyData.java:187) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor92.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: java.util.concurrent.ExecutionException: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:130) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > ... 14 more >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:15,189 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.GetStoragePoolInfoVDSCommand] (DefaultQuartzScheduler_Worker-95) Command GetStoragePoolInfoVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = true) execution failed. Exception: IRSProtocolException: IRSProtocolException: >2014-10-19 12:43:16,205 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-19 12:43:18,205 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:18,205 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:18,207 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-6) Exception during connection: java.util.concurrent.ExecutionException: java.rmi.ConnectException: Connection timeout > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:122) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:18,210 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-6) Command GetCapabilitiesVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c]) execution failed. Exception: VDSNetworkException: java.rmi.ConnectException: Connection timeout >2014-10-19 12:43:18,210 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Host alma06.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-10-19 12:43:18,219 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-6) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.rmi.ConnectException: Connection timeout > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:21,229 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-19 12:43:23,229 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:23,229 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:23,230 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-53) Exception during connection: java.util.concurrent.ExecutionException: java.rmi.ConnectException: Connection timeout > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:122) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:23,233 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-53) Command GetCapabilitiesVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c]) execution failed. Exception: VDSNetworkException: java.rmi.ConnectException: Connection timeout >2014-10-19 12:43:23,233 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-53) Host alma06.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-10-19 12:43:23,242 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-53) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.rmi.ConnectException: Connection timeout > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:25,194 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-19 12:43:27,194 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:27,194 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:27,194 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-19 12:43:27,194 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-25) Exception during connection: java.util.concurrent.ExecutionException: java.rmi.ConnectException: Connection timeout > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:122) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.spmStatus(JsonRpcVdsServer.java:794) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand.executeVdsBrokerCommand(SpmStatusVDSCommand.java:24) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:418) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData.proceedStoragePoolStats(IrsProxyData.java:221) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData._updatingTimer_Elapsed(IrsProxyData.java:187) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor92.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:27,198 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-25) Command SpmStatusVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916) execution failed. Exception: VDSNetworkException: java.rmi.ConnectException: Connection timeout >2014-10-19 12:43:29,195 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:29,195 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:29,195 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-19 12:43:29,195 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-9) Exception during connection: java.util.concurrent.ExecutionException: java.rmi.ConnectException: Connection timeout > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:122) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:29,201 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-9) Command GetCapabilitiesVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c]) execution failed. Exception: VDSNetworkException: java.rmi.ConnectException: Connection timeout >2014-10-19 12:43:29,201 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-9) Host alma06.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-10-19 12:43:29,209 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-9) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.rmi.ConnectException: Connection timeout > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:31,196 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:31,196 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:31,196 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-25) Exception during connection: java.util.concurrent.ExecutionException: java.rmi.ConnectException: Connection timeout > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:122) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcIIrsServer.getStoragePoolInfo(JsonRpcIIrsServer.java:381) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.GetStoragePoolInfoVDSCommand.executeIrsBrokerCommand(GetStoragePoolInfoVDSCommand.java:31) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand.executeVDSCommand(IrsBrokerCommand.java:156) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:418) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData.proceedStoragePoolStats(IrsProxyData.java:293) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData._updatingTimer_Elapsed(IrsProxyData.java:187) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor92.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:31,199 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-25) IrsBroker::Failed::GetStoragePoolInfoVDS >2014-10-19 12:43:31,199 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-25) ERROR, org.ovirt.engine.core.vdsbroker.irsbroker.GetStoragePoolInfoVDSCommand, exception: XmlRpcRunTimeException: Connection issues during send request, log id: 1338f3eb: org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcRunTimeException: Connection issues during send request > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:62) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcIIrsServer.getStoragePoolInfo(JsonRpcIIrsServer.java:381) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.GetStoragePoolInfoVDSCommand.executeIrsBrokerCommand(GetStoragePoolInfoVDSCommand.java:31) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand.executeVDSCommand(IrsBrokerCommand.java:156) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:418) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData.proceedStoragePoolStats(IrsProxyData.java:293) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData._updatingTimer_Elapsed(IrsProxyData.java:187) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor92.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: java.util.concurrent.ExecutionException: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:130) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > ... 14 more >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:31,200 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.GetStoragePoolInfoVDSCommand] (DefaultQuartzScheduler_Worker-25) Command GetStoragePoolInfoVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = true) execution failed. Exception: IRSProtocolException: IRSProtocolException: >2014-10-19 12:43:32,217 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-19 12:43:34,218 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:34,218 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:34,219 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-86) Exception during connection: java.util.concurrent.ExecutionException: java.rmi.ConnectException: Connection timeout > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:122) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:34,222 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-86) Command GetCapabilitiesVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c]) execution failed. Exception: VDSNetworkException: java.rmi.ConnectException: Connection timeout >2014-10-19 12:43:34,222 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-86) Host alma06.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-10-19 12:43:34,230 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-86) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.rmi.ConnectException: Connection timeout > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:37,239 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-19 12:43:37,392 ERROR [org.ovirt.engine.core.bll.SshSoftFencingCommand] (org.ovirt.thread.pool-7-thread-30) [787e1d51] SSH Soft Fencing command failed on host alma06.qa.lab.tlv.redhat.com: Failed to get the session. >Stdout: >Stderr: >Stacktrace: org.apache.sshd.common.RuntimeSshException: Failed to get the session.: org.apache.sshd.common.RuntimeSshException: Failed to get the session. > at org.apache.sshd.client.future.DefaultConnectFuture.getSession(DefaultConnectFuture.java:43) > at org.ovirt.engine.core.uutils.ssh.SSHClient.connect(SSHClient.java:294) [uutils.jar:] > at org.ovirt.engine.core.bll.utils.EngineSSHClient.connect(EngineSSHClient.java:59) [bll.jar:] > at org.ovirt.engine.core.bll.SshSoftFencingCommand.executeSshSoftFencingCommand(SshSoftFencingCommand.java:87) [bll.jar:] > at org.ovirt.engine.core.bll.SshSoftFencingCommand.executeCommand(SshSoftFencingCommand.java:53) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1168) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1307) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1932) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1331) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:344) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:430) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:411) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:616) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor128.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52) [jboss-as-ee.jar:7.4.1.Final-redhat-3] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) [jboss-as-ee.jar:7.4.1.Final-redhat-3] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:74) [jboss-as-weld.jar:7.4.1.Final-redhat-3] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:84) [jboss-as-weld.jar:7.4.1.Final-redhat-3] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:97) [jboss-as-weld.jar:7.4.1.Final-redhat-3] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) [jboss-as-ee.jar:7.4.1.Final-redhat-3] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) [jboss-as-ee.jar:7.4.1.Final-redhat-3] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [jboss-as-ejb3.jar:7.4.1.Final-redhat-3] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:93) [jboss-as-weld.jar:7.4.1.Final-redhat-3] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee.jar:7.4.1.Final-redhat-3] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3.jar:7.4.1.Final-redhat-3] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:259) [jboss-as-ejb3.jar:7.4.1.Final-redhat-3] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:398) [jboss-as-ejb3.jar:7.4.1.Final-redhat-3] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:242) [jboss-as-ejb3.jar:7.4.1.Final-redhat-3] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3.jar:7.4.1.Final-redhat-3] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3.jar:7.4.1.Final-redhat-3] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3.jar:7.4.1.Final-redhat-3] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee.jar:7.4.1.Final-redhat-3] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee.jar:7.4.1.Final-redhat-3] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:185) [jboss-as-ee.jar:7.4.1.Final-redhat-3] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee.jar:7.4.1.Final-redhat-3] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73) [jboss-as-ee.jar:7.4.1.Final-redhat-3] > at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view8.runInternalAction(Unknown Source) [bll.jar:] > at org.ovirt.engine.core.bll.VdsEventListener$4.run(VdsEventListener.java:249) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:90) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_65] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_65] > at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_65] >Caused by: java.net.ConnectException: Connection timed out > at sun.nio.ch.UnixAsynchronousSocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.UnixAsynchronousSocketChannelImpl.finishConnect(UnixAsynchronousSocketChannelImpl.java:252) [rt.jar:1.7.0_65] > at sun.nio.ch.UnixAsynchronousSocketChannelImpl.finish(UnixAsynchronousSocketChannelImpl.java:198) [rt.jar:1.7.0_65] > at sun.nio.ch.UnixAsynchronousSocketChannelImpl.onEvent(UnixAsynchronousSocketChannelImpl.java:213) [rt.jar:1.7.0_65] > at sun.nio.ch.EPollPort$EventHandlerTask.run(EPollPort.java:293) [rt.jar:1.7.0_65] > ... 1 more > >2014-10-19 12:43:37,429 INFO [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-7-thread-30) [2a43e27c] Lock Acquired to object EngineLock [exclusiveLocks= key: 1c39bd88-8672-408f-a593-60d6ef8f647c value: VDS_FENCE >, sharedLocks= ] >2014-10-19 12:43:37,429 INFO [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-7-thread-30) [2a43e27c] Fence failed on vds alma06.qa.lab.tlv.redhat.com which is spm of pool f8f5eaee-8fd0-4b45-87db-62d61b03a916 - moving pool to non operational >2014-10-19 12:43:37,451 WARN [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (org.ovirt.thread.pool-7-thread-30) [362bbfea] The message key SetStoragePoolStatus is missing from bundles/ExecutionMessages >2014-10-19 12:43:37,528 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (org.ovirt.thread.pool-7-thread-30) [362bbfea] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: f8f5eaee-8fd0-4b45-87db-62d61b03a916 Type: StoragePool >2014-10-19 12:43:37,538 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-30) [362bbfea] Correlation ID: 362bbfea, Job ID: 4367a12e-d307-42b2-83de-e68e0583bf5e, Call Stack: null, Custom Event ID: -1, Message: Fencing failed on Storage Pool Manager alma06.qa.lab.tlv.redhat.com for Data Center dc_35. Setting status to Non-Operational. >2014-10-19 12:43:37,543 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (org.ovirt.thread.pool-7-thread-30) [362bbfea] Lock freed to object EngineLock [exclusiveLocks= key: 1c39bd88-8672-408f-a593-60d6ef8f647c value: VDS_FENCE >, sharedLocks= ] >2014-10-19 12:43:37,543 ERROR [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-7-thread-30) [362bbfea] Failed to run Fence script on vds:alma06.qa.lab.tlv.redhat.com. >2014-10-19 12:43:37,551 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-30) [362bbfea] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com became non responsive. It has no power management configured. Please check the host status, manually reboot it, and click "Confirm Host Has Been Rebooted" >2014-10-19 12:43:37,551 WARN [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-7-thread-30) [362bbfea] CanDoAction of action VdsNotRespondingTreatment failed. Reasons:VAR__ACTION__RESTART,VDS_FENCE_DISABLED >2014-10-19 12:43:37,551 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-30) [362bbfea] Trying to release exclusive lock which does not exist, lock key: 1c39bd88-8672-408f-a593-60d6ef8f647cVDS_FENCE >2014-10-19 12:43:37,551 INFO [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-7-thread-30) [362bbfea] Lock freed to object EngineLock [exclusiveLocks= key: 1c39bd88-8672-408f-a593-60d6ef8f647c value: VDS_FENCE >, sharedLocks= ] >2014-10-19 12:43:39,240 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:39,240 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:39,241 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-72) Exception during connection: java.util.concurrent.ExecutionException: java.rmi.ConnectException: Connection timeout > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:122) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:39,244 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-72) Command GetCapabilitiesVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c]) execution failed. Exception: VDSNetworkException: java.rmi.ConnectException: Connection timeout >2014-10-19 12:43:39,244 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-72) Host alma06.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-10-19 12:43:39,252 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-72) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.rmi.ConnectException: Connection timeout > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:42,264 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-19 12:43:44,264 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:44,264 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:44,264 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-39) Exception during connection: java.util.concurrent.ExecutionException: java.rmi.ConnectException: Connection timeout > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:122) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:44,267 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-39) Command GetCapabilitiesVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c]) execution failed. Exception: VDSNetworkException: java.rmi.ConnectException: Connection timeout >2014-10-19 12:43:44,267 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-39) Host alma06.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-10-19 12:43:44,276 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-39) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.rmi.ConnectException: Connection timeout > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:47,285 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-19 12:43:49,285 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:49,285 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:49,285 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-98) Exception during connection: java.util.concurrent.ExecutionException: java.rmi.ConnectException: Connection timeout > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:122) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:49,288 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-98) Command GetCapabilitiesVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c]) execution failed. Exception: VDSNetworkException: java.rmi.ConnectException: Connection timeout >2014-10-19 12:43:49,288 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-98) Host alma06.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-10-19 12:43:49,297 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-98) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.rmi.ConnectException: Connection timeout > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:52,306 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-19 12:43:54,306 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:54,306 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:54,306 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-99) Exception during connection: java.util.concurrent.ExecutionException: java.rmi.ConnectException: Connection timeout > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:122) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:54,309 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-99) Command GetCapabilitiesVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c]) execution failed. Exception: VDSNetworkException: java.rmi.ConnectException: Connection timeout >2014-10-19 12:43:54,309 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-99) Host alma06.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-10-19 12:43:54,319 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-99) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.rmi.ConnectException: Connection timeout > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:57,331 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-19 12:43:59,332 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:59,332 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:59,332 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-9) Exception during connection: java.util.concurrent.ExecutionException: java.rmi.ConnectException: Connection timeout > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:122) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:43:59,335 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-9) Command GetCapabilitiesVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c]) execution failed. Exception: VDSNetworkException: java.rmi.ConnectException: Connection timeout >2014-10-19 12:43:59,336 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-9) Host alma06.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-10-19 12:43:59,345 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-9) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.rmi.ConnectException: Connection timeout > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:02,353 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-19 12:44:04,354 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:04,354 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:04,354 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-11) Exception during connection: java.util.concurrent.ExecutionException: java.rmi.ConnectException: Connection timeout > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:122) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:04,357 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-11) [43a5b796] Command GetCapabilitiesVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c]) execution failed. Exception: VDSNetworkException: java.rmi.ConnectException: Connection timeout >2014-10-19 12:44:04,357 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-11) [43a5b796] Host alma06.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-10-19 12:44:04,365 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-11) [43a5b796] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.rmi.ConnectException: Connection timeout > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:07,376 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-19 12:44:09,376 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:09,376 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:09,376 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-26) Exception during connection: java.util.concurrent.ExecutionException: java.rmi.ConnectException: Connection timeout > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:122) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:09,379 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-26) Command GetCapabilitiesVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c]) execution failed. Exception: VDSNetworkException: java.rmi.ConnectException: Connection timeout >2014-10-19 12:44:09,380 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-26) Host alma06.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-10-19 12:44:09,388 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-26) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.rmi.ConnectException: Connection timeout > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:12,397 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-19 12:44:14,397 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:14,397 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:14,397 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-22) Exception during connection: java.util.concurrent.ExecutionException: java.rmi.ConnectException: Connection timeout > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:122) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:14,400 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-22) Command GetCapabilitiesVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c]) execution failed. Exception: VDSNetworkException: java.rmi.ConnectException: Connection timeout >2014-10-19 12:44:14,401 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-22) Host alma06.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-10-19 12:44:14,408 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-22) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.rmi.ConnectException: Connection timeout > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:17,418 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-19 12:44:19,418 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:19,418 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:19,419 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-58) Exception during connection: java.util.concurrent.ExecutionException: java.rmi.ConnectException: Connection timeout > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:122) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:19,422 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-58) [53ccdc22] Command GetCapabilitiesVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c]) execution failed. Exception: VDSNetworkException: java.rmi.ConnectException: Connection timeout >2014-10-19 12:44:19,422 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-58) [53ccdc22] Host alma06.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-10-19 12:44:19,430 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-58) [53ccdc22] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.rmi.ConnectException: Connection timeout > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:22,441 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-19 12:44:24,442 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:24,442 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:24,443 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-66) Exception during connection: java.util.concurrent.ExecutionException: java.rmi.ConnectException: Connection timeout > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:122) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:24,446 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-66) Command GetCapabilitiesVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c]) execution failed. Exception: VDSNetworkException: java.rmi.ConnectException: Connection timeout >2014-10-19 12:44:24,447 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-66) Host alma06.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-10-19 12:44:24,455 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-66) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.rmi.ConnectException: Connection timeout > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:27,467 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-19 12:44:29,467 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:29,467 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:29,467 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-2) Exception during connection: java.util.concurrent.ExecutionException: java.rmi.ConnectException: Connection timeout > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:122) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:29,470 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-2) Command GetCapabilitiesVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c]) execution failed. Exception: VDSNetworkException: java.rmi.ConnectException: Connection timeout >2014-10-19 12:44:29,470 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-2) Host alma06.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-10-19 12:44:29,479 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-2) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.rmi.ConnectException: Connection timeout > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:32,488 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-19 12:44:33,489 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:111) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:33,490 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:111) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:33,490 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-89) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:122) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:111) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:33,493 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-89) Command GetCapabilitiesVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-10-19 12:44:33,493 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-89) Host alma06.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-10-19 12:44:33,502 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-89) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:111) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:36,512 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-19 12:44:36,513 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:111) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:36,513 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:111) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:36,514 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-26) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:122) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:111) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:36,517 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-26) Command GetCapabilitiesVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-10-19 12:44:36,517 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-26) Host alma06.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-10-19 12:44:36,526 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-26) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:111) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:39,535 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-19 12:44:39,536 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:111) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:39,536 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:111) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:39,537 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-35) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:122) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:111) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:39,540 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-35) Command GetCapabilitiesVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-10-19 12:44:39,540 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-35) Host alma06.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-10-19 12:44:39,568 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-35) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:111) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:42,574 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-19 12:44:42,575 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:111) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:42,575 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:111) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:42,575 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-48) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:122) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:111) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:42,578 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-48) Command GetCapabilitiesVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-10-19 12:44:42,578 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-48) Host alma06.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-10-19 12:44:42,586 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-48) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:111) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:45,596 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-19 12:44:45,597 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:111) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:45,597 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:111) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:45,597 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-60) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:122) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:111) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:45,600 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-60) Command GetCapabilitiesVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-10-19 12:44:45,601 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-60) Host alma06.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-10-19 12:44:45,609 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-60) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:111) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:48,619 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-19 12:44:50,619 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:50,619 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:50,619 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-34) Exception during connection: java.util.concurrent.ExecutionException: java.rmi.ConnectException: Connection timeout > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:122) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:50,622 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-34) Command GetCapabilitiesVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c]) execution failed. Exception: VDSNetworkException: java.rmi.ConnectException: Connection timeout >2014-10-19 12:44:50,622 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-34) Host alma06.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-10-19 12:44:50,631 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-34) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.rmi.ConnectException: Connection timeout > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:53,642 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-19 12:44:55,642 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:55,642 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:55,643 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-5) Exception during connection: java.util.concurrent.ExecutionException: java.rmi.ConnectException: Connection timeout > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:122) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:55,645 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-5) Command GetCapabilitiesVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c]) execution failed. Exception: VDSNetworkException: java.rmi.ConnectException: Connection timeout >2014-10-19 12:44:55,646 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-5) Host alma06.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-10-19 12:44:55,655 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-5) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.rmi.ConnectException: Connection timeout > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:44:58,666 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-19 12:45:00,666 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:45:00,666 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:45:00,666 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-15) Exception during connection: java.util.concurrent.ExecutionException: java.rmi.ConnectException: Connection timeout > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:122) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:45:00,669 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-15) Command GetCapabilitiesVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c]) execution failed. Exception: VDSNetworkException: java.rmi.ConnectException: Connection timeout >2014-10-19 12:45:00,670 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-15) Host alma06.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-10-19 12:45:00,678 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-15) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.rmi.ConnectException: Connection timeout > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.rmi.ConnectException: Connection timeout > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:113) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-19 12:45:03,688 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-19 12:45:09,281 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-97) START, GetHardwareInfoVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c]), log id: 3da2de93 >2014-10-19 12:45:09,371 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-97) FINISH, GetHardwareInfoVDSCommand, log id: 3da2de93 >2014-10-19 12:45:09,399 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-97) [1a1c0a46] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 1c39bd88-8672-408f-a593-60d6ef8f647c Type: VDS >2014-10-19 12:45:09,408 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-97) [1a1c0a46] VM iscsi_vm (686605cb-13f6-41b7-bd14-04ae79b55799) is running in db and not running in VDS alma06.qa.lab.tlv.redhat.com >2014-10-19 12:45:09,410 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-97) [35c47f94] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: f8f5eaee-8fd0-4b45-87db-62d61b03a916 Type: StoragePool >2014-10-19 12:45:09,418 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-97) [5dfff26] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: f8f5eaee-8fd0-4b45-87db-62d61b03a916 Type: StoragePool >2014-10-19 12:45:09,423 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-97) [5dfff26] START, ConnectStorageServerVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, storageType = NFS, connectionList = [{ id: 5eec92db-0201-4aa5-8598-1fe041c7a057, connection: 10.35.160.108:/RHEV/artyom_storage_rhevm_35, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 7b457326 >2014-10-19 12:45:09,699 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-97) [5dfff26] FINISH, ConnectStorageServerVDSCommand, return: {5eec92db-0201-4aa5-8598-1fe041c7a057=0}, log id: 7b457326 >2014-10-19 12:45:09,699 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-97) [5dfff26] Host alma06.qa.lab.tlv.redhat.com storage connection was succeeded >2014-10-19 12:45:09,707 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-7-thread-43) START, ConnectStoragePoolVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vdsId = 1c39bd88-8672-408f-a593-60d6ef8f647c, storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, masterVersion = 1), log id: 550c91bd >2014-10-19 12:45:10,059 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-7-thread-43) FINISH, ConnectStoragePoolVDSCommand, log id: 550c91bd >2014-10-19 12:45:10,082 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-97) [5dfff26] START, SetMOMPolicyParametersVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c), log id: 2aca8446 >2014-10-19 12:45:10,165 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-97) [5dfff26] FINISH, SetMOMPolicyParametersVDSCommand, log id: 2aca8446 >2014-10-19 12:45:10,171 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-97) [5dfff26] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host alma06.qa.lab.tlv.redhat.com was set to Up. >2014-10-19 12:45:10,179 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-97) [5dfff26] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host alma06.qa.lab.tlv.redhat.com. >2014-10-19 12:45:10,199 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-97) [3e37d958] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 1c39bd88-8672-408f-a593-60d6ef8f647c Type: VDS >2014-10-19 12:45:10,270 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (org.ovirt.thread.pool-7-thread-50) [759c2b7c] Running command: ProcessDownVmCommand internal: true. >2014-10-19 12:45:11,515 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (DefaultQuartzScheduler_Worker-48) Command org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand return value > >TaskStatusListReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=654, mMessage=Not SPM]] > >2014-10-19 12:45:11,516 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (DefaultQuartzScheduler_Worker-48) HostName = alma06.qa.lab.tlv.redhat.com >2014-10-19 12:45:11,516 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (DefaultQuartzScheduler_Worker-48) Command HSMGetAllTasksStatusesVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c) execution failed. Exception: IRSNonOperationalException: IRSGenericException: IRSErrorException: IRSNonOperationalException: Not SPM >2014-10-19 12:45:11,555 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-48) hostFromVds::selectedVds - alma06.qa.lab.tlv.redhat.com, spmStatus Free, storage pool dc_35 >2014-10-19 12:45:11,586 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-48) SpmStatus on vds 1c39bd88-8672-408f-a593-60d6ef8f647c: Free >2014-10-19 12:45:11,591 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-48) starting spm on vds alma06.qa.lab.tlv.redhat.com, storage pool dc_35, prevId 4, LVER 10 >2014-10-19 12:45:11,594 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-48) START, SpmStartVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, prevId=4, prevLVER=10, storagePoolFormatType=V3, recoveryMode=Manual, SCSIFencing=false), log id: 49abd387 >2014-10-19 12:45:11,607 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-48) spmStart polling started: taskId = bc9d4c7e-c51f-4721-b35d-d60bbf03e306 >2014-10-19 12:47:51,936 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-48) spmStart polling ended: taskId = bc9d4c7e-c51f-4721-b35d-d60bbf03e306 task status = finished >2014-10-19 12:47:51,954 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-48) spmStart polling ended, spm status: SPM >2014-10-19 12:47:51,955 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (DefaultQuartzScheduler_Worker-48) START, HSMClearTaskVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, taskId=bc9d4c7e-c51f-4721-b35d-d60bbf03e306), log id: 32098c69 >2014-10-19 12:47:51,962 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (DefaultQuartzScheduler_Worker-48) FINISH, HSMClearTaskVDSCommand, log id: 32098c69 >2014-10-19 12:47:51,962 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-48) FINISH, SpmStartVDSCommand, return: org.ovirt.engine.core.common.businessentities.SpmStatusResult@1af9de4e, log id: 49abd387 >2014-10-19 12:47:51,965 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-48) Initialize Irs proxy from vds: alma06.qa.lab.tlv.redhat.com >2014-10-19 12:47:51,970 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-48) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Storage Pool Manager runs on Host alma06.qa.lab.tlv.redhat.com (Address: alma06.qa.lab.tlv.redhat.com). >2014-10-19 12:47:51,979 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-36) START, SPMGetAllTasksInfoVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false), log id: 51c23b7c >2014-10-19 12:47:52,105 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-19 12:47:52,284 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-36) -- executeIrsBrokerCommand: Attempting on storage pool f8f5eaee-8fd0-4b45-87db-62d61b03a916 >2014-10-19 12:47:52,285 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-36) START, HSMGetAllTasksInfoVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c), log id: 62f14427 >2014-10-19 12:47:52,292 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-36) FINISH, HSMGetAllTasksInfoVDSCommand, return: [], log id: 62f14427 >2014-10-19 12:47:52,292 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-36) FINISH, SPMGetAllTasksInfoVDSCommand, return: [], log id: 51c23b7c >2014-10-19 12:47:52,292 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-7-thread-36) Discovered no tasks on Storage Pool dc_35 >2014-10-19 12:59:30,266 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-71) Attempting to update VMs/Templates Ovf. >2014-10-19 13:10:35,215 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-83) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-19 13:35:15,199 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-10) Running command: LoginUserCommand internal: false. >2014-10-19 13:35:15,214 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-10-19 13:35:37,465 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (ajp-/127.0.0.1:8702-6) Running command: LoginAdminUserCommand internal: false. >2014-10-19 13:35:37,469 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-10-19 13:35:37,652 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-1) Running command: LoginUserCommand internal: false. >2014-10-19 13:36:09,998 ERROR [org.ovirt.engine.core.utils.servlet.ServletUtils] (ajp-/127.0.0.1:8702-11) Can't read file "/etc/ovirt-engine/branding/50-rhev-2.brand/images/dialog/info.png" for request "/ovirt-engine/webadmin/theme/50-rhev-2.brand/images/dialog/info.png", will send a 404 error response. >2014-10-19 13:36:10,966 INFO [org.ovirt.engine.core.bll.ShutdownVmCommand] (org.ovirt.thread.pool-7-thread-12) [df91781] Running command: ShutdownVmCommand internal: false. Entities affected : ID: 9236151d-8fb4-4c9a-830a-53af2b584dc7 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-10-19 13:36:10,971 INFO [org.ovirt.engine.core.bll.ShutdownVmCommand] (org.ovirt.thread.pool-7-thread-12) [df91781] Entered (VM test_vm). >2014-10-19 13:36:10,971 INFO [org.ovirt.engine.core.bll.ShutdownVmCommand] (org.ovirt.thread.pool-7-thread-12) [df91781] Sending shutdown command for VM test_vm. >2014-10-19 13:36:10,981 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (org.ovirt.thread.pool-7-thread-12) [df91781] START, DestroyVmVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, vmId=9236151d-8fb4-4c9a-830a-53af2b584dc7, force=false, secondsToWait=30, gracefully=true, reason=), log id: 4400e412 >2014-10-19 13:36:11,049 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-7-thread-12) [df91781] START, DestroyVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, vmId=9236151d-8fb4-4c9a-830a-53af2b584dc7, force=false, secondsToWait=30, gracefully=true, reason=), log id: 27db9e1b >2014-10-19 13:36:11,058 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-7-thread-12) [df91781] FINISH, DestroyVDSCommand, log id: 27db9e1b >2014-10-19 13:36:11,063 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (org.ovirt.thread.pool-7-thread-12) [df91781] FINISH, DestroyVmVDSCommand, return: PoweringDown, log id: 4400e412 >2014-10-19 13:36:11,072 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-12) [df91781] Correlation ID: df91781, Job ID: d182feea-cf22-40d4-8002-fb29aa388a8a, Call Stack: null, Custom Event ID: -1, Message: VM shutdown initiated by admin on VM test_vm (Host: cyan-vdsg.qa.lab.tlv.redhat.com) (Reason: Not Specified). >2014-10-19 13:36:14,002 INFO [org.ovirt.engine.core.bll.ShutdownVmCommand] (org.ovirt.thread.pool-7-thread-22) [79615e77] Running command: ShutdownVmCommand internal: false. Entities affected : ID: 9236151d-8fb4-4c9a-830a-53af2b584dc7 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-10-19 13:36:14,004 INFO [org.ovirt.engine.core.bll.ShutdownVmCommand] (org.ovirt.thread.pool-7-thread-22) [79615e77] Entered (VM test_vm). >2014-10-19 13:36:14,004 INFO [org.ovirt.engine.core.bll.ShutdownVmCommand] (org.ovirt.thread.pool-7-thread-22) [79615e77] Cannot shutdown VM test_vm, status is not up. Stopping instead. >2014-10-19 13:36:14,071 INFO [org.ovirt.engine.core.bll.StopVmCommand] (org.ovirt.thread.pool-7-thread-22) [2af8c4b5] Running command: StopVmCommand internal: true. Entities affected : ID: 9236151d-8fb4-4c9a-830a-53af2b584dc7 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-10-19 13:36:14,076 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (org.ovirt.thread.pool-7-thread-22) [2af8c4b5] START, DestroyVmVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, vmId=9236151d-8fb4-4c9a-830a-53af2b584dc7, force=false, secondsToWait=0, gracefully=false, reason=), log id: 6b03f32c >2014-10-19 13:36:14,084 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-7-thread-22) [2af8c4b5] START, DestroyVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, vmId=9236151d-8fb4-4c9a-830a-53af2b584dc7, force=false, secondsToWait=0, gracefully=false, reason=), log id: 2ddeef54 >2014-10-19 13:36:14,728 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-7-thread-22) [2af8c4b5] FINISH, DestroyVDSCommand, log id: 2ddeef54 >2014-10-19 13:36:14,733 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (org.ovirt.thread.pool-7-thread-22) [2af8c4b5] FINISH, DestroyVmVDSCommand, return: Down, log id: 6b03f32c >2014-10-19 13:36:14,805 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-22) [2af8c4b5] Correlation ID: 2af8c4b5, Job ID: 5689e472-8b12-4d85-a436-df90e54c0ecc, Call Stack: null, Custom Event ID: -1, Message: VM test_vm was powered off ungracefully by admin (Host: cyan-vdsg.qa.lab.tlv.redhat.com) (Reason: Not Specified). >2014-10-19 13:36:14,812 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (org.ovirt.thread.pool-7-thread-10) [59e89564] Running command: ProcessDownVmCommand internal: true. >2014-10-19 13:36:18,465 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-97) [3e37d958] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-19 13:36:31,677 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp-/127.0.0.1:8702-4) [7a74034a] Lock Acquired to object EngineLock [exclusiveLocks= key: test_vm value: VM_NAME >, sharedLocks= key: 9236151d-8fb4-4c9a-830a-53af2b584dc7 value: VM >] >2014-10-19 13:36:31,840 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp-/127.0.0.1:8702-4) [7a74034a] Running command: UpdateVmCommand internal: false. Entities affected : ID: 9236151d-8fb4-4c9a-830a-53af2b584dc7 Type: VMAction group EDIT_VM_PROPERTIES with role type USER >2014-10-19 13:36:31,865 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [7a74034a] Correlation ID: 7a74034a, Job ID: e36c44ee-7500-4673-b7b2-fb1b53f9f3de, Call Stack: null, Custom Event ID: -1, Message: VM test_vm configuration was updated by admin. >2014-10-19 13:36:31,869 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp-/127.0.0.1:8702-4) [7a74034a] Lock freed to object EngineLock [exclusiveLocks= key: test_vm value: VM_NAME >, sharedLocks= key: 9236151d-8fb4-4c9a-830a-53af2b584dc7 value: VM >] >2014-10-19 13:37:23,117 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp-/127.0.0.1:8702-5) [4c643745] Lock Acquired to object EngineLock [exclusiveLocks= key: test_vm value: VM_NAME >, sharedLocks= key: 9236151d-8fb4-4c9a-830a-53af2b584dc7 value: VM >] >2014-10-19 13:37:23,173 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp-/127.0.0.1:8702-5) [4c643745] Running command: UpdateVmCommand internal: false. Entities affected : ID: 9236151d-8fb4-4c9a-830a-53af2b584dc7 Type: VMAction group EDIT_VM_PROPERTIES with role type USER, ID: 9236151d-8fb4-4c9a-830a-53af2b584dc7 Type: VMAction group EDIT_ADMIN_VM_PROPERTIES with role type ADMIN >2014-10-19 13:37:23,190 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [4c643745] Correlation ID: 4c643745, Job ID: d0c9a91f-8d8b-46a6-931b-779ac55abb13, Call Stack: null, Custom Event ID: -1, Message: VM test_vm configuration was updated by admin. >2014-10-19 13:37:23,193 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp-/127.0.0.1:8702-5) [4c643745] Lock freed to object EngineLock [exclusiveLocks= key: test_vm value: VM_NAME >, sharedLocks= key: 9236151d-8fb4-4c9a-830a-53af2b584dc7 value: VM >] >2014-10-19 13:37:32,814 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp-/127.0.0.1:8702-3) [1e8f3a69] Lock Acquired to object EngineLock [exclusiveLocks= key: test_vm value: VM_NAME >, sharedLocks= key: 9236151d-8fb4-4c9a-830a-53af2b584dc7 value: VM >] >2014-10-19 13:37:32,855 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp-/127.0.0.1:8702-3) [1e8f3a69] Running command: UpdateVmCommand internal: false. Entities affected : ID: 9236151d-8fb4-4c9a-830a-53af2b584dc7 Type: VMAction group EDIT_VM_PROPERTIES with role type USER >2014-10-19 13:37:32,875 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) [1e8f3a69] Correlation ID: 1e8f3a69, Job ID: 6f96691a-df16-4aa9-bde6-0b391c035ea6, Call Stack: null, Custom Event ID: -1, Message: VM test_vm configuration was updated by admin. >2014-10-19 13:37:32,878 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp-/127.0.0.1:8702-3) [1e8f3a69] Lock freed to object EngineLock [exclusiveLocks= key: test_vm value: VM_NAME >, sharedLocks= key: 9236151d-8fb4-4c9a-830a-53af2b584dc7 value: VM >] >2014-10-19 13:59:23,936 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-28) Deactivating non existing principal admin::internal >2014-10-19 13:59:30,288 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-17) Attempting to update VMs/Templates Ovf. >2014-10-19 13:59:30,442 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-17) [15d2f2f9] Lock Acquired to object EngineLock [exclusiveLocks= key: 1742b7af-bb81-4e6b-bf10-feab68a2f799 value: DISK >key: a1341057-9cf8-4750-a734-656d05a5d2c5 value: STORAGE >key: dfba228f-46f3-4758-a4f0-05857683a088 value: DISK >, sharedLocks= key: f8f5eaee-8fd0-4b45-87db-62d61b03a916 value: OVF_UPDATE >] >2014-10-19 13:59:30,446 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-17) [15d2f2f9] Running command: ProcessOvfUpdateForStorageDomainCommand internal: true. Entities affected : ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: Storage >2014-10-19 13:59:30,469 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-17) [15d2f2f9] START, SetVolumeDescriptionVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = dfba228f-46f3-4758-a4f0-05857683a088, imageId = 26155adf-9478-4927-8c49-8e12088c8648), log id: 60142c69 >2014-10-19 13:59:30,639 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-17) [15d2f2f9] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-10-19 13:59:30,639 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-17) [15d2f2f9] -- setVolumeDescription parameters: > spUUID=f8f5eaee-8fd0-4b45-87db-62d61b03a916 > sdUUID=a1341057-9cf8-4750-a734-656d05a5d2c5 > imageGroupGUID=dfba228f-46f3-4758-a4f0-05857683a088 > volUUID=26155adf-9478-4927-8c49-8e12088c8648 > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"a1341057-9cf8-4750-a734-656d05a5d2c5"}],"Last Updated":"Tue Oct 14 16:59:24 IDT 2014"} > >2014-10-19 13:59:30,745 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-17) [15d2f2f9] FINISH, SetVolumeDescriptionVDSCommand, log id: 60142c69 >2014-10-19 13:59:30,784 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-17) [38b6f487] Running command: UploadStreamCommand internal: true. Entities affected : ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: Storage >2014-10-19 13:59:30,787 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-17) [38b6f487] START, UploadStreamVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c), log id: 64ad88fa >2014-10-19 13:59:30,787 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-17) [38b6f487] -- executeVdsBrokerCommand: >2014-10-19 13:59:30,787 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-17) [38b6f487] -- parameters: > spUUID=f8f5eaee-8fd0-4b45-87db-62d61b03a916 > sdUUID=a1341057-9cf8-4750-a734-656d05a5d2c5 > imageGUID=dfba228f-46f3-4758-a4f0-05857683a088 > volUUID=26155adf-9478-4927-8c49-8e12088c8648 > size=61,440 > >2014-10-19 13:59:31,120 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-17) [38b6f487] FINISH, UploadStreamVDSCommand, log id: 64ad88fa >2014-10-19 13:59:31,122 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-17) [38b6f487] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 63b1e7ae-0ce2-4bd6-9acd-9130ed47f3c6 >2014-10-19 13:59:31,122 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-17) [38b6f487] CommandMultiAsyncTasks::AttachTask: Attaching task 3740d123-db74-4d83-9ec8-af311e3635bc to command 63b1e7ae-0ce2-4bd6-9acd-9130ed47f3c6. >2014-10-19 13:59:31,136 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-17) [38b6f487] Adding task 3740d123-db74-4d83-9ec8-af311e3635bc (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-10-19 13:59:31,144 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-17) [38b6f487] Lock freed to object EngineLock [exclusiveLocks= key: 1742b7af-bb81-4e6b-bf10-feab68a2f799 value: DISK >key: a1341057-9cf8-4750-a734-656d05a5d2c5 value: STORAGE >key: dfba228f-46f3-4758-a4f0-05857683a088 value: DISK >, sharedLocks= key: f8f5eaee-8fd0-4b45-87db-62d61b03a916 value: OVF_UPDATE >] >2014-10-19 13:59:31,150 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-17) [38b6f487] START, SetVolumeDescriptionVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = dfba228f-46f3-4758-a4f0-05857683a088, imageId = 26155adf-9478-4927-8c49-8e12088c8648), log id: 63984a5c >2014-10-19 13:59:31,150 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-17) [38b6f487] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-10-19 13:59:31,151 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-17) [38b6f487] -- setVolumeDescription parameters: > spUUID=f8f5eaee-8fd0-4b45-87db-62d61b03a916 > sdUUID=a1341057-9cf8-4750-a734-656d05a5d2c5 > imageGroupGUID=dfba228f-46f3-4758-a4f0-05857683a088 > volUUID=26155adf-9478-4927-8c49-8e12088c8648 > description={"Updated":true,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"a1341057-9cf8-4750-a734-656d05a5d2c5"}],"Last Updated":"Sun Oct 19 13:59:30 IDT 2014","Size":61440} > >2014-10-19 13:59:31,196 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-17) [38b6f487] FINISH, SetVolumeDescriptionVDSCommand, log id: 63984a5c >2014-10-19 13:59:31,223 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-17) [38b6f487] START, SetVolumeDescriptionVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = 1742b7af-bb81-4e6b-bf10-feab68a2f799, imageId = 5b2b66c8-425d-4c49-9b3c-62f7586aac1b), log id: 1332639a >2014-10-19 13:59:31,223 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-17) [38b6f487] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-10-19 13:59:31,223 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-17) [38b6f487] -- setVolumeDescription parameters: > spUUID=f8f5eaee-8fd0-4b45-87db-62d61b03a916 > sdUUID=a1341057-9cf8-4750-a734-656d05a5d2c5 > imageGroupGUID=1742b7af-bb81-4e6b-bf10-feab68a2f799 > volUUID=5b2b66c8-425d-4c49-9b3c-62f7586aac1b > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"a1341057-9cf8-4750-a734-656d05a5d2c5"}],"Last Updated":"Tue Oct 14 16:59:24 IDT 2014"} > >2014-10-19 13:59:31,267 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-17) [38b6f487] FINISH, SetVolumeDescriptionVDSCommand, log id: 1332639a >2014-10-19 13:59:31,291 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-17) [34aca7ff] Running command: UploadStreamCommand internal: true. Entities affected : ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: Storage >2014-10-19 13:59:31,293 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-17) [34aca7ff] START, UploadStreamVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c), log id: 703e6072 >2014-10-19 13:59:31,293 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-17) [34aca7ff] -- executeVdsBrokerCommand: >2014-10-19 13:59:31,293 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-17) [34aca7ff] -- parameters: > spUUID=f8f5eaee-8fd0-4b45-87db-62d61b03a916 > sdUUID=a1341057-9cf8-4750-a734-656d05a5d2c5 > imageGUID=1742b7af-bb81-4e6b-bf10-feab68a2f799 > volUUID=5b2b66c8-425d-4c49-9b3c-62f7586aac1b > size=61,440 > >2014-10-19 13:59:32,475 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-17) [34aca7ff] FINISH, UploadStreamVDSCommand, log id: 703e6072 >2014-10-19 13:59:32,477 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-17) [34aca7ff] CommandMultiAsyncTasks::AttachTask: Attaching task 3284afae-7fdc-49cf-a5de-133686f4bfc2 to command 63b1e7ae-0ce2-4bd6-9acd-9130ed47f3c6. >2014-10-19 13:59:32,489 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-17) [34aca7ff] Adding task 3284afae-7fdc-49cf-a5de-133686f4bfc2 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-10-19 13:59:32,494 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-17) [34aca7ff] Trying to release a shared lock for key: f8f5eaee-8fd0-4b45-87db-62d61b03a916OVF_UPDATE , but lock does not exist >2014-10-19 13:59:32,494 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-17) [34aca7ff] Trying to release exclusive lock which does not exist, lock key: 1742b7af-bb81-4e6b-bf10-feab68a2f799DISK >2014-10-19 13:59:32,495 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-17) [34aca7ff] Trying to release exclusive lock which does not exist, lock key: a1341057-9cf8-4750-a734-656d05a5d2c5STORAGE >2014-10-19 13:59:32,495 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-17) [34aca7ff] Trying to release exclusive lock which does not exist, lock key: dfba228f-46f3-4758-a4f0-05857683a088DISK >2014-10-19 13:59:32,495 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-17) [34aca7ff] Lock freed to object EngineLock [exclusiveLocks= key: 1742b7af-bb81-4e6b-bf10-feab68a2f799 value: DISK >key: a1341057-9cf8-4750-a734-656d05a5d2c5 value: STORAGE >key: dfba228f-46f3-4758-a4f0-05857683a088 value: DISK >, sharedLocks= key: f8f5eaee-8fd0-4b45-87db-62d61b03a916 value: OVF_UPDATE >] >2014-10-19 13:59:32,497 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-17) [34aca7ff] START, SetVolumeDescriptionVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = 1742b7af-bb81-4e6b-bf10-feab68a2f799, imageId = 5b2b66c8-425d-4c49-9b3c-62f7586aac1b), log id: 59542d68 >2014-10-19 13:59:32,497 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-17) [34aca7ff] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-10-19 13:59:32,497 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-17) [34aca7ff] -- setVolumeDescription parameters: > spUUID=f8f5eaee-8fd0-4b45-87db-62d61b03a916 > sdUUID=a1341057-9cf8-4750-a734-656d05a5d2c5 > imageGroupGUID=1742b7af-bb81-4e6b-bf10-feab68a2f799 > volUUID=5b2b66c8-425d-4c49-9b3c-62f7586aac1b > description={"Updated":true,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"a1341057-9cf8-4750-a734-656d05a5d2c5"}],"Last Updated":"Sun Oct 19 13:59:30 IDT 2014","Size":61440} > >2014-10-19 13:59:32,981 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-17) [34aca7ff] FINISH, SetVolumeDescriptionVDSCommand, log id: 59542d68 >2014-10-19 13:59:32,989 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-17) [34aca7ff] BaseAsyncTask::startPollingTask: Starting to poll task 3740d123-db74-4d83-9ec8-af311e3635bc. >2014-10-19 13:59:32,989 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-17) [34aca7ff] BaseAsyncTask::startPollingTask: Starting to poll task 3284afae-7fdc-49cf-a5de-133686f4bfc2. >2014-10-19 13:59:42,786 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-42) Polling and updating Async Tasks: 2 tasks, 2 tasks to poll now >2014-10-19 13:59:42,796 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-42) SPMAsyncTask::PollTask: Polling task 3284afae-7fdc-49cf-a5de-133686f4bfc2 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-10-19 13:59:42,796 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-42) BaseAsyncTask::onTaskEndSuccess: Task 3284afae-7fdc-49cf-a5de-133686f4bfc2 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-10-19 13:59:42,797 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-42) Task with DB Task ID f402b983-0902-461f-9cf2-0f11635ec06b and VDSM Task ID 3740d123-db74-4d83-9ec8-af311e3635bc is in state Polling. End action for command 63b1e7ae-0ce2-4bd6-9acd-9130ed47f3c6 will proceed when all the entitys tasks are completed. >2014-10-19 13:59:42,797 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-42) SPMAsyncTask::PollTask: Polling task 3740d123-db74-4d83-9ec8-af311e3635bc (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-10-19 13:59:42,797 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-42) BaseAsyncTask::onTaskEndSuccess: Task 3740d123-db74-4d83-9ec8-af311e3635bc (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-10-19 13:59:42,797 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-42) CommandAsyncTask::endActionIfNecessary: All tasks of command 63b1e7ae-0ce2-4bd6-9acd-9130ed47f3c6 has ended -> executing endAction >2014-10-19 13:59:42,797 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-42) CommandAsyncTask::endAction: Ending action for 2 tasks (command ID: 63b1e7ae-0ce2-4bd6-9acd-9130ed47f3c6): calling endAction . >2014-10-19 13:59:42,798 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-30) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction ProcessOvfUpdateForStorageDomain, executionIndex: 0 >2014-10-19 13:59:42,811 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (org.ovirt.thread.pool-7-thread-30) [15d2f2f9] Ending command successfully: org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand >2014-10-19 13:59:42,812 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-30) [15d2f2f9] Trying to release a shared lock for key: f8f5eaee-8fd0-4b45-87db-62d61b03a916OVF_UPDATE , but lock does not exist >2014-10-19 13:59:42,812 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-30) [15d2f2f9] Trying to release exclusive lock which does not exist, lock key: 1742b7af-bb81-4e6b-bf10-feab68a2f799DISK >2014-10-19 13:59:42,812 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-30) [15d2f2f9] Trying to release exclusive lock which does not exist, lock key: a1341057-9cf8-4750-a734-656d05a5d2c5STORAGE >2014-10-19 13:59:42,812 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-30) [15d2f2f9] Trying to release exclusive lock which does not exist, lock key: dfba228f-46f3-4758-a4f0-05857683a088DISK >2014-10-19 13:59:42,812 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (org.ovirt.thread.pool-7-thread-30) [15d2f2f9] Lock freed to object EngineLock [exclusiveLocks= key: 1742b7af-bb81-4e6b-bf10-feab68a2f799 value: DISK >key: a1341057-9cf8-4750-a734-656d05a5d2c5 value: STORAGE >key: dfba228f-46f3-4758-a4f0-05857683a088 value: DISK >, sharedLocks= key: f8f5eaee-8fd0-4b45-87db-62d61b03a916 value: OVF_UPDATE >] >2014-10-19 13:59:42,812 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-30) [15d2f2f9] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type ProcessOvfUpdateForStorageDomain completed, handling the result. >2014-10-19 13:59:42,812 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-30) [15d2f2f9] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type ProcessOvfUpdateForStorageDomain succeeded, clearing tasks. >2014-10-19 13:59:42,812 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-30) [15d2f2f9] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 3284afae-7fdc-49cf-a5de-133686f4bfc2 >2014-10-19 13:59:42,813 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-30) [15d2f2f9] START, SPMClearTaskVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, taskId = 3284afae-7fdc-49cf-a5de-133686f4bfc2), log id: 4ca41133 >2014-10-19 13:59:42,814 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-30) [15d2f2f9] START, HSMClearTaskVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, taskId=3284afae-7fdc-49cf-a5de-133686f4bfc2), log id: 27655294 >2014-10-19 13:59:42,845 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-30) [15d2f2f9] FINISH, HSMClearTaskVDSCommand, log id: 27655294 >2014-10-19 13:59:42,845 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-30) [15d2f2f9] FINISH, SPMClearTaskVDSCommand, log id: 4ca41133 >2014-10-19 13:59:42,850 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-30) [15d2f2f9] BaseAsyncTask::removeTaskFromDB: Removed task 3284afae-7fdc-49cf-a5de-133686f4bfc2 from DataBase >2014-10-19 13:59:42,850 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-30) [15d2f2f9] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 3740d123-db74-4d83-9ec8-af311e3635bc >2014-10-19 13:59:42,851 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-30) [15d2f2f9] START, SPMClearTaskVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, taskId = 3740d123-db74-4d83-9ec8-af311e3635bc), log id: 13b81ea6 >2014-10-19 13:59:42,852 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-30) [15d2f2f9] START, HSMClearTaskVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, taskId=3740d123-db74-4d83-9ec8-af311e3635bc), log id: 1ce42e67 >2014-10-19 13:59:42,873 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-30) [15d2f2f9] FINISH, HSMClearTaskVDSCommand, log id: 1ce42e67 >2014-10-19 13:59:42,873 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-30) [15d2f2f9] FINISH, SPMClearTaskVDSCommand, log id: 13b81ea6 >2014-10-19 13:59:42,877 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-30) [15d2f2f9] BaseAsyncTask::removeTaskFromDB: Removed task 3740d123-db74-4d83-9ec8-af311e3635bc from DataBase >2014-10-19 13:59:42,877 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-30) [15d2f2f9] CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 63b1e7ae-0ce2-4bd6-9acd-9130ed47f3c6 >2014-10-19 14:00:59,406 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-38) Setting new tasks map. The map contains now 0 tasks >2014-10-19 14:00:59,407 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-38) Cleared all tasks of pool f8f5eaee-8fd0-4b45-87db-62d61b03a916. >2014-10-19 14:10:37,852 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-94) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-19 14:36:22,051 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-62) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 418 seconds while maximum configured value is 300 seconds. >2014-10-19 14:59:32,990 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-100) Attempting to update VMs/Templates Ovf. >2014-10-19 15:10:43,062 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-62) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-19 15:36:24,983 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-3) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-19 15:59:33,006 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-52) [7b116c2f] Attempting to update VMs/Templates Ovf. >2014-10-19 16:10:47,229 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-76) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-19 16:36:28,739 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-49) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-19 16:45:48,535 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp-/127.0.0.1:8702-11) [cbf3eb6] Lock Acquired to object EngineLock [exclusiveLocks= key: 686605cb-13f6-41b7-bd14-04ae79b55799 value: VM >, sharedLocks= ] >2014-10-19 16:45:48,569 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp-/127.0.0.1:8702-11) [cbf3eb6] START, IsVmDuringInitiatingVDSCommand( vmId = 686605cb-13f6-41b7-bd14-04ae79b55799), log id: 7fdebffa >2014-10-19 16:45:48,569 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp-/127.0.0.1:8702-11) [cbf3eb6] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 7fdebffa >2014-10-19 16:45:48,582 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ajp-/127.0.0.1:8702-11) [cbf3eb6] Candidate host cyan-vdsg.qa.lab.tlv.redhat.com (3a2a45bd-4411-4428-aa45-a514d30d3ea6) was filtered out by VAR__FILTERTYPE__INTERNAL filter Memory >2014-10-19 16:45:48,709 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-7-thread-39) [cbf3eb6] Running command: RunVmCommand internal: false. Entities affected : ID: 686605cb-13f6-41b7-bd14-04ae79b55799 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-10-19 16:45:48,745 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (org.ovirt.thread.pool-7-thread-39) [cbf3eb6] Candidate host cyan-vdsg.qa.lab.tlv.redhat.com (3a2a45bd-4411-4428-aa45-a514d30d3ea6) was filtered out by VAR__FILTERTYPE__INTERNAL filter Memory (correlation id: cbf3eb6) >2014-10-19 16:45:48,786 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-7-thread-39) [cbf3eb6] START, UpdateVmDynamicDataVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@1061674c), log id: 3e3df954 >2014-10-19 16:45:48,791 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-7-thread-39) [cbf3eb6] FINISH, UpdateVmDynamicDataVDSCommand, log id: 3e3df954 >2014-10-19 16:45:48,796 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-7-thread-39) [cbf3eb6] START, CreateVmVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vmId=686605cb-13f6-41b7-bd14-04ae79b55799, vm=VM [iscsi_vm]), log id: 3de7e7d9 >2014-10-19 16:45:48,807 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-7-thread-39) [cbf3eb6] START, CreateVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vmId=686605cb-13f6-41b7-bd14-04ae79b55799, vm=VM [iscsi_vm]), log id: 786745f1 >2014-10-19 16:45:48,848 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-7-thread-39) [cbf3eb6] org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand kvmEnable=true,keyboardLayout=en-us,nice=0,displayNetwork=rhevm,copyPasteEnable=true,timeOffset=0,transparentHugePages=true,vmId=686605cb-13f6-41b7-bd14-04ae79b55799,acpiEnable=true,custom={device_c120de5c-6eca-42ca-b378-36a1d13c38acdevice_b4140cd5-4a59-4db3-a15b-c731ef60690b=VmDevice {vmId=686605cb-13f6-41b7-bd14-04ae79b55799, deviceId=b4140cd5-4a59-4db3-a15b-c731ef60690b, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={bus=0, controller=0, type=virtio-serial, port=1}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}, snapshotId=null}, device_c120de5c-6eca-42ca-b378-36a1d13c38ac=VmDevice {vmId=686605cb-13f6-41b7-bd14-04ae79b55799, deviceId=c120de5c-6eca-42ca-b378-36a1d13c38ac, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={slot=0x01, bus=0x00, domain=0x0000, type=pci, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}, snapshotId=null}, device_c120de5c-6eca-42ca-b378-36a1d13c38acdevice_b4140cd5-4a59-4db3-a15b-c731ef60690bdevice_a078be42-04c4-4622-8834-fb9dc3eebf43=VmDevice {vmId=686605cb-13f6-41b7-bd14-04ae79b55799, deviceId=a078be42-04c4-4622-8834-fb9dc3eebf43, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={bus=0, controller=0, type=virtio-serial, port=2}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}, snapshotId=null}, device_c120de5c-6eca-42ca-b378-36a1d13c38acdevice_b4140cd5-4a59-4db3-a15b-c731ef60690bdevice_a078be42-04c4-4622-8834-fb9dc3eebf43device_e5a13326-1702-48dd-8a46-f8a8eaed7fac=VmDevice {vmId=686605cb-13f6-41b7-bd14-04ae79b55799, deviceId=e5a13326-1702-48dd-8a46-f8a8eaed7fac, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={bus=0, controller=0, type=virtio-serial, port=3}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}, snapshotId=null}},spiceSslCipherSuite=DEFAULT,memSize=4096,smp=4,emulatedMachine=rhel6.5.0,vmType=kvm,memGuaranteedSize=4096,display=qxl,smartcardEnable=false,bootMenuEnable=false,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,smpCoresPerSocket=2,maxVCpus=32,devices=[{address={bus=0x00, domain=0x0000, type=pci, slot=0x02, function=0x0}, specParams={vram=32768, heads=1}, device=qxl, type=video, deviceId=0b8359e8-4300-4dd3-b70a-1d5b86e36a2b}, {shared=false, iface=ide, index=2, address={unit=0, bus=1, target=0, controller=0, type=drive}, specParams={path=}, path=, device=cdrom, type=disk, readonly=true, deviceId=e11ab561-1197-4a6c-8d8a-7b5cb0420fe5}, {shared=false, index=0, volumeID=ee2380f5-cb1b-48b4-9e3a-d8e730e4b50f, propagateErrors=off, format=raw, type=disk, iface=virtio, bootOrder=2, address={bus=0x00, domain=0x0000, type=pci, slot=0x06, function=0x0}, domainID=a1341057-9cf8-4750-a734-656d05a5d2c5, imageID=2cc3f4d2-4b42-49b6-9260-5b604d5ffa81, specParams={}, optional=false, device=disk, poolID=f8f5eaee-8fd0-4b45-87db-62d61b03a916, readonly=false, deviceId=2cc3f4d2-4b42-49b6-9260-5b604d5ffa81}, {shared=false, volumeID=c0983994-564d-4c02-8260-d3acb02251fb, iface=virtio, imageID=f988c27d-db38-4446-b082-2c4107280333, domainID=a1341057-9cf8-4750-a734-656d05a5d2c5, address={bus=0x00, domain=0x0000, type=pci, slot=0x07, function=0x0}, specParams={}, optional=false, propagateErrors=off, device=disk, poolID=f8f5eaee-8fd0-4b45-87db-62d61b03a916, format=raw, type=disk, readonly=false, deviceId=f988c27d-db38-4446-b082-2c4107280333}, {nicModel=pv, bootOrder=1, address={bus=0x00, domain=0x0000, type=pci, slot=0x03, function=0x0}, specParams={outbound={}, inbound={}}, macAddr=00:1a:4a:91:91:01, device=bridge, linkActive=true, type=interface, filter=vdsm-no-mac-spoofing, network=rhevm, deviceId=8a6b85f8-5db4-4b77-83ee-73f565277ca4}, {address={bus=0x00, domain=0x0000, type=pci, slot=0x08, function=0x0}, specParams={model=virtio}, device=memballoon, type=balloon, deviceId=f757cedb-3ab7-4a15-979f-a97fb0a9d2dd}, {index=0, model=virtio-scsi, address={bus=0x00, domain=0x0000, type=pci, slot=0x04, function=0x0}, specParams={}, device=scsi, type=controller, deviceId=d61b5bbc-cf3d-45f1-a00d-d9ba1275f7fd}, {address={bus=0x00, domain=0x0000, type=pci, slot=0x05, function=0x0}, specParams={}, device=virtio-serial, type=controller, deviceId=18f5350d-597e-4325-b6af-7b6485678225}],vmName=iscsi_vm,cpuType=Opteron_G1,fileTransferEnable=true >2014-10-19 16:45:48,958 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-7-thread-39) [cbf3eb6] FINISH, CreateVDSCommand, log id: 786745f1 >2014-10-19 16:45:48,964 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-7-thread-39) [cbf3eb6] FINISH, CreateVmVDSCommand, return: WaitForLaunch, log id: 3de7e7d9 >2014-10-19 16:45:48,964 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-7-thread-39) [cbf3eb6] Lock freed to object EngineLock [exclusiveLocks= key: 686605cb-13f6-41b7-bd14-04ae79b55799 value: VM >, sharedLocks= ] >2014-10-19 16:45:48,969 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-39) [cbf3eb6] Correlation ID: cbf3eb6, Job ID: a295cc20-2b77-40e3-9a7f-f811bed7d048, Call Stack: null, Custom Event ID: -1, Message: VM iscsi_vm was started by admin (Host: alma06.qa.lab.tlv.redhat.com). >2014-10-19 16:45:49,060 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-71) START, FullListVdsCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c], vmIds=[686605cb-13f6-41b7-bd14-04ae79b55799]), log id: 453fe8cb >2014-10-19 16:45:49,073 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-71) FINISH, FullListVdsCommand, return: [{acpiEnable=true, emulatedMachine=rhel6.5.0, pid=0, memGuaranteedSize=4096, transparentHugePages=true, displaySecurePort=-1, spiceSslCipherSuite=DEFAULT, cpuType=Opteron_G1, smp=4, custom={device_c120de5c-6eca-42ca-b378-36a1d13c38acdevice_b4140cd5-4a59-4db3-a15b-c731ef60690b=VmDevice {vmId=686605cb-13f6-41b7-bd14-04ae79b55799, deviceId=b4140cd5-4a59-4db3-a15b-c731ef60690b, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={bus=0, controller=0, type=virtio-serial, port=1}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}, snapshotId=null}, device_c120de5c-6eca-42ca-b378-36a1d13c38ac=VmDevice {vmId=686605cb-13f6-41b7-bd14-04ae79b55799, deviceId=c120de5c-6eca-42ca-b378-36a1d13c38ac, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={slot=0x01, bus=0x00, domain=0x0000, type=pci, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}, snapshotId=null}, device_c120de5c-6eca-42ca-b378-36a1d13c38acdevice_b4140cd5-4a59-4db3-a15b-c731ef60690bdevice_a078be42-04c4-4622-8834-fb9dc3eebf43=VmDevice {vmId=686605cb-13f6-41b7-bd14-04ae79b55799, deviceId=a078be42-04c4-4622-8834-fb9dc3eebf43, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={bus=0, controller=0, type=virtio-serial, port=2}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}, snapshotId=null}, device_c120de5c-6eca-42ca-b378-36a1d13c38acdevice_b4140cd5-4a59-4db3-a15b-c731ef60690bdevice_a078be42-04c4-4622-8834-fb9dc3eebf43device_e5a13326-1702-48dd-8a46-f8a8eaed7fac=VmDevice {vmId=686605cb-13f6-41b7-bd14-04ae79b55799, deviceId=e5a13326-1702-48dd-8a46-f8a8eaed7fac, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={bus=0, controller=0, type=virtio-serial, port=3}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}, snapshotId=null}}, vmType=kvm, memSize=4096, smpCoresPerSocket=2, vmName=iscsi_vm, nice=0, status=WaitForLaunch, bootMenuEnable=false, vmId=686605cb-13f6-41b7-bd14-04ae79b55799, copyPasteEnable=true, displayIp=10.35.117.30, displayPort=-1, smartcardEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, fileTransferEnable=true, nicModel=rtl8139,pv, keyboardLayout=en-us, kvmEnable=true, displayNetwork=rhevm, devices=[Ljava.lang.Object;@49faed38, timeOffset=0, maxVCpus=32, clientIp=, display=qxl}], log id: 453fe8cb >2014-10-19 16:45:49,075 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-71) Received a spice Device without an address when processing VM 686605cb-13f6-41b7-bd14-04ae79b55799 devices, skipping device: {device=spice, specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, copyPasteEnable=true, keyMap=en-us, displayIp=10.35.117.30, displayNetwork=rhevm}, deviceType=graphics, type=graphics} >2014-10-19 16:45:52,139 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-81) VM iscsi_vm 686605cb-13f6-41b7-bd14-04ae79b55799 moved from WaitForLaunch --> PoweringUp >2014-10-19 16:45:52,144 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-81) START, FullListVdsCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c], vmIds=[686605cb-13f6-41b7-bd14-04ae79b55799]), log id: 5cd07193 >2014-10-19 16:45:52,160 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-81) FINISH, FullListVdsCommand, return: [{acpiEnable=true, emulatedMachine=rhel6.5.0, pid=28281, memGuaranteedSize=4096, transparentHugePages=true, displaySecurePort=5901, spiceSslCipherSuite=DEFAULT, cpuType=Opteron_G1, smp=4, pauseCode=NOERR, custom={device_c120de5c-6eca-42ca-b378-36a1d13c38acdevice_b4140cd5-4a59-4db3-a15b-c731ef60690b=VmDevice {vmId=686605cb-13f6-41b7-bd14-04ae79b55799, deviceId=b4140cd5-4a59-4db3-a15b-c731ef60690b, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={bus=0, controller=0, type=virtio-serial, port=1}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}, snapshotId=null}, device_c120de5c-6eca-42ca-b378-36a1d13c38ac=VmDevice {vmId=686605cb-13f6-41b7-bd14-04ae79b55799, deviceId=c120de5c-6eca-42ca-b378-36a1d13c38ac, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={slot=0x01, bus=0x00, domain=0x0000, type=pci, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}, snapshotId=null}, device_c120de5c-6eca-42ca-b378-36a1d13c38acdevice_b4140cd5-4a59-4db3-a15b-c731ef60690bdevice_a078be42-04c4-4622-8834-fb9dc3eebf43=VmDevice {vmId=686605cb-13f6-41b7-bd14-04ae79b55799, deviceId=a078be42-04c4-4622-8834-fb9dc3eebf43, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={bus=0, controller=0, type=virtio-serial, port=2}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}, snapshotId=null}, device_c120de5c-6eca-42ca-b378-36a1d13c38acdevice_b4140cd5-4a59-4db3-a15b-c731ef60690bdevice_a078be42-04c4-4622-8834-fb9dc3eebf43device_e5a13326-1702-48dd-8a46-f8a8eaed7fac=VmDevice {vmId=686605cb-13f6-41b7-bd14-04ae79b55799, deviceId=e5a13326-1702-48dd-8a46-f8a8eaed7fac, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={bus=0, controller=0, type=virtio-serial, port=3}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}, snapshotId=null}}, vmType=kvm, memSize=4096, smpCoresPerSocket=2, vmName=iscsi_vm, nice=0, status=Up, bootMenuEnable=false, vmId=686605cb-13f6-41b7-bd14-04ae79b55799, copyPasteEnable=true, displayIp=10.35.117.30, displayPort=5900, smartcardEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, fileTransferEnable=true, nicModel=rtl8139,pv, keyboardLayout=en-us, kvmEnable=true, displayNetwork=rhevm, devices=[Ljava.lang.Object;@78d0997f, timeOffset=0, maxVCpus=32, clientIp=, display=qxl}], log id: 5cd07193 >2014-10-19 16:45:52,162 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-81) Received a spice Device without an address when processing VM 686605cb-13f6-41b7-bd14-04ae79b55799 devices, skipping device: {tlsPort=5901, specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, copyPasteEnable=true, keyMap=en-us, displayIp=10.35.117.30, displayNetwork=rhevm}, deviceType=graphics, device=spice, type=graphics, port=5900} >2014-10-19 16:46:49,433 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (ajp-/127.0.0.1:8702-4) [14a140d4] Running command: SetVmTicketCommand internal: false. Entities affected : ID: 686605cb-13f6-41b7-bd14-04ae79b55799 Type: VMAction group CONNECT_TO_VM with role type USER >2014-10-19 16:46:49,441 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-4) [14a140d4] START, SetVmTicketVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vmId=686605cb-13f6-41b7-bd14-04ae79b55799, ticket=XsA+LM1XRoNu, validTime=120,m userName=admin, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 38e84cff >2014-10-19 16:46:49,460 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-4) [14a140d4] FINISH, SetVmTicketVDSCommand, log id: 38e84cff >2014-10-19 16:46:49,485 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [14a140d4] Correlation ID: 14a140d4, Call Stack: null, Custom Event ID: -1, Message: user admin initiated console session for VM iscsi_vm >2014-10-19 16:46:49,925 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-87) VM iscsi_vm 686605cb-13f6-41b7-bd14-04ae79b55799 moved from PoweringUp --> Up >2014-10-19 16:46:49,934 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-87) Correlation ID: cbf3eb6, Job ID: a295cc20-2b77-40e3-9a7f-f811bed7d048, Call Stack: null, Custom Event ID: -1, Message: VM iscsi_vm started on Host alma06.qa.lab.tlv.redhat.com >2014-10-19 16:46:59,101 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-27) [55c797c1] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin is connected to VM iscsi_vm. >2014-10-19 16:48:09,103 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-72) VM iscsi_vm 686605cb-13f6-41b7-bd14-04ae79b55799 moved from Up --> RebootInProgress >2014-10-19 16:48:09,107 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-72) START, FullListVdsCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c], vmIds=[686605cb-13f6-41b7-bd14-04ae79b55799]), log id: 2f91709f >2014-10-19 16:48:09,126 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-72) FINISH, FullListVdsCommand, return: [{acpiEnable=true, emulatedMachine=rhel6.5.0, pid=28281, memGuaranteedSize=4096, transparentHugePages=true, displaySecurePort=5901, spiceSslCipherSuite=DEFAULT, cpuType=Opteron_G1, smp=4, pauseCode=NOERR, custom={device_c120de5c-6eca-42ca-b378-36a1d13c38acdevice_b4140cd5-4a59-4db3-a15b-c731ef60690b=VmDevice {vmId=686605cb-13f6-41b7-bd14-04ae79b55799, deviceId=b4140cd5-4a59-4db3-a15b-c731ef60690b, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={bus=0, controller=0, type=virtio-serial, port=1}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}, snapshotId=null}, device_c120de5c-6eca-42ca-b378-36a1d13c38ac=VmDevice {vmId=686605cb-13f6-41b7-bd14-04ae79b55799, deviceId=c120de5c-6eca-42ca-b378-36a1d13c38ac, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={slot=0x01, bus=0x00, domain=0x0000, type=pci, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}, snapshotId=null}, device_c120de5c-6eca-42ca-b378-36a1d13c38acdevice_b4140cd5-4a59-4db3-a15b-c731ef60690bdevice_a078be42-04c4-4622-8834-fb9dc3eebf43=VmDevice {vmId=686605cb-13f6-41b7-bd14-04ae79b55799, deviceId=a078be42-04c4-4622-8834-fb9dc3eebf43, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={bus=0, controller=0, type=virtio-serial, port=2}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}, snapshotId=null}, device_c120de5c-6eca-42ca-b378-36a1d13c38acdevice_b4140cd5-4a59-4db3-a15b-c731ef60690bdevice_a078be42-04c4-4622-8834-fb9dc3eebf43device_e5a13326-1702-48dd-8a46-f8a8eaed7fac=VmDevice {vmId=686605cb-13f6-41b7-bd14-04ae79b55799, deviceId=e5a13326-1702-48dd-8a46-f8a8eaed7fac, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={bus=0, controller=0, type=virtio-serial, port=3}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}, snapshotId=null}}, vmType=kvm, memSize=4096, smpCoresPerSocket=2, vmName=iscsi_vm, nice=0, status=Up, bootMenuEnable=false, vmId=686605cb-13f6-41b7-bd14-04ae79b55799, copyPasteEnable=true, displayIp=10.35.117.30, displayPort=5900, smartcardEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, fileTransferEnable=true, nicModel=rtl8139,pv, keyboardLayout=en-us, kvmEnable=true, displayNetwork=rhevm, devices=[Ljava.lang.Object;@7578ef1e, timeOffset=0, maxVCpus=32, clientIp=10.35.4.67, display=qxl}], log id: 2f91709f >2014-10-19 16:48:09,130 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-72) Received a spice Device without an address when processing VM 686605cb-13f6-41b7-bd14-04ae79b55799 devices, skipping device: {tlsPort=5901, specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, copyPasteEnable=true, keyMap=en-us, displayIp=10.35.117.30, displayNetwork=rhevm}, deviceType=graphics, device=spice, type=graphics, port=5900} >2014-10-19 16:49:09,937 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-95) VM iscsi_vm 686605cb-13f6-41b7-bd14-04ae79b55799 moved from RebootInProgress --> Up >2014-10-19 16:51:47,979 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-31) [31332b57] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin got disconnected from VM iscsi_vm. >2014-10-19 16:59:33,021 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-11) [43a5b796] Attempting to update VMs/Templates Ovf. >2014-10-19 17:10:57,142 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-81) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-19 17:36:31,503 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-81) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-19 17:59:33,035 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-16) Attempting to update VMs/Templates Ovf. >2014-10-19 18:11:01,108 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-54) [3a38f005] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-19 18:36:37,218 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-95) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-19 18:59:33,050 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-78) Attempting to update VMs/Templates Ovf. >2014-10-19 19:11:05,272 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-60) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-19 19:36:42,157 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-19 19:59:33,065 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-92) Attempting to update VMs/Templates Ovf. >2014-10-19 20:11:07,908 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-57) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-19 20:36:48,471 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-100) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-19 20:59:33,080 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-81) Attempting to update VMs/Templates Ovf. >2014-10-19 21:11:11,906 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-42) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-19 21:36:52,412 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-82) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-19 21:59:33,095 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-76) Attempting to update VMs/Templates Ovf. >2014-10-19 22:11:16,709 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-34) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-19 22:36:57,821 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-67) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-19 22:59:33,109 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-94) Attempting to update VMs/Templates Ovf. >2014-10-19 23:11:20,088 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-100) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-19 23:37:01,814 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-10) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-19 23:59:33,123 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-82) Attempting to update VMs/Templates Ovf. >2014-10-20 00:11:24,277 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-10) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 00:37:05,835 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-60) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 00:59:33,139 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-11) [43a5b796] Attempting to update VMs/Templates Ovf. >2014-10-20 01:11:26,715 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-83) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 01:37:10,171 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-27) [55c797c1] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 01:59:33,153 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-9) Attempting to update VMs/Templates Ovf. >2014-10-20 02:11:29,840 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-99) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 02:37:14,267 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-61) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 02:59:33,167 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-46) Attempting to update VMs/Templates Ovf. >2014-10-20 03:11:33,302 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-63) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 03:35:35,001 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-64) Start deleteAgedOutAuditLogs >2014-10-20 03:35:35,001 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-4) Start deleteAgedOutCommandEntities >2014-10-20 03:35:35,004 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-4) Finished deleteAgedOutCommandEntities >2014-10-20 03:35:35,012 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-64) Finished deleteAgedOutAuditLogs >2014-10-20 03:37:21,362 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-17) [34aca7ff] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 03:59:33,183 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-1) Attempting to update VMs/Templates Ovf. >2014-10-20 04:11:36,700 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-8) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 04:37:28,793 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-35) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 04:59:33,201 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-85) Attempting to update VMs/Templates Ovf. >2014-10-20 05:11:39,645 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-44) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 05:37:36,113 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-17) [34aca7ff] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 05:59:33,214 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-28) Attempting to update VMs/Templates Ovf. >2014-10-20 06:11:42,184 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 06:37:43,484 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-13) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 06:59:33,229 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-37) Attempting to update VMs/Templates Ovf. >2014-10-20 07:11:45,565 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) [43a5b796] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 07:37:49,094 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-90) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 07:59:33,244 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-33) Attempting to update VMs/Templates Ovf. >2014-10-20 08:11:49,115 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-52) [7b116c2f] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 08:37:54,742 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-54) [3a38f005] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 08:59:33,258 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-2) Attempting to update VMs/Templates Ovf. >2014-10-20 09:11:52,499 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-47) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 09:37:59,537 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-62) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 09:59:33,273 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-31) [31332b57] Attempting to update VMs/Templates Ovf. >2014-10-20 10:11:56,441 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 10:38:06,043 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-76) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 10:59:33,287 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-90) Attempting to update VMs/Templates Ovf. >2014-10-20 11:12:01,150 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-78) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 11:38:11,804 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-97) [3e37d958] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 11:59:33,304 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-95) Attempting to update VMs/Templates Ovf. >2014-10-20 12:06:13,919 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-11) Running command: LoginUserCommand internal: false. >2014-10-20 12:06:13,922 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-10-20 12:07:12,226 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-4) [1be9eea8] Running command: LogoutUserCommand internal: false. >2014-10-20 12:07:12,226 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-1) [1b3365e9] Running command: LogoutUserCommand internal: false. >2014-10-20 12:07:12,228 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-6) [6ab9d1bb] Running command: LogoutUserCommand internal: false. >2014-10-20 12:07:12,229 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-5) [1385751a] Running command: LogoutUserCommand internal: false. >2014-10-20 12:07:12,230 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-10) [21dfb756] Running command: LogoutUserCommand internal: false. >2014-10-20 12:07:12,232 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-11) [7456d964] Running command: LogoutUserCommand internal: false. >2014-10-20 12:07:12,234 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) [6ab9d1bb] Correlation ID: 6ab9d1bb, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-10-20 12:07:12,236 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [1b3365e9] Correlation ID: 1b3365e9, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-10-20 12:07:12,238 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [1385751a] Correlation ID: 1385751a, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-10-20 12:07:12,239 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) [7456d964] Correlation ID: 7456d964, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-10-20 12:07:12,239 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) [21dfb756] Correlation ID: 21dfb756, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-10-20 12:07:12,241 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [1be9eea8] Correlation ID: 1be9eea8, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-10-20 12:07:12,250 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-2) [7667165b] Running command: LogoutUserCommand internal: false. >2014-10-20 12:07:12,258 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-6) Invalid locale found in configuration: >2014-10-20 12:07:12,258 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-6) Invalid locale found in configuration: >2014-10-20 12:07:12,260 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) [7667165b] Correlation ID: 7667165b, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-10-20 12:12:05,947 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-36) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 12:14:17,308 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (ajp-/127.0.0.1:8702-6) Running command: LoginAdminUserCommand internal: false. >2014-10-20 12:14:17,320 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-10-20 12:15:20,758 INFO [org.ovirt.engine.core.bll.AddVdsGroupCommand] (ajp-/127.0.0.1:8702-7) [56966b00] Running command: AddVdsGroupCommand internal: false. Entities affected : ID: f8f5eaee-8fd0-4b45-87db-62d61b03a916 Type: StoragePoolAction group CREATE_CLUSTER with role type ADMIN >2014-10-20 12:15:20,786 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [56966b00] Correlation ID: 56966b00, Call Stack: null, Custom Event ID: -1, Message: Cluster cl_35_ppc was added by admin >2014-10-20 12:31:16,365 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (ajp-/127.0.0.1:8702-1) [1f208773] Lock Acquired to object EngineLock [exclusiveLocks= key: 4f2a4935-4de5-44a1-845d-06dfbe33e003 value: VDS >, sharedLocks= ] >2014-10-20 12:31:16,374 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (org.ovirt.thread.pool-7-thread-33) [1f208773] Running command: RemoveVdsCommand internal: false. Entities affected : ID: 4f2a4935-4de5-44a1-845d-06dfbe33e003 Type: VDSAction group DELETE_HOST with role type ADMIN >2014-10-20 12:31:16,486 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (org.ovirt.thread.pool-7-thread-33) [1f208773] START, RemoveVdsVDSCommand( HostId = 4f2a4935-4de5-44a1-845d-06dfbe33e003), log id: 56a337b4 >2014-10-20 12:31:16,486 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (org.ovirt.thread.pool-7-thread-33) [1f208773] vdsManager::disposing >2014-10-20 12:31:16,486 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (org.ovirt.thread.pool-7-thread-33) [1f208773] FINISH, RemoveVdsVDSCommand, log id: 56a337b4 >2014-10-20 12:31:16,493 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-33) [1f208773] Correlation ID: 1f208773, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com was removed by admin. >2014-10-20 12:31:16,494 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (org.ovirt.thread.pool-7-thread-33) [1f208773] Lock freed to object EngineLock [exclusiveLocks= key: 4f2a4935-4de5-44a1-845d-06dfbe33e003 value: VDS >, sharedLocks= ] >2014-10-20 12:31:38,638 INFO [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-9) [2064bac5] Running command: AddVdsCommand internal: false. Entities affected : ID: 63702093-c04f-417c-8045-aa56334b54ee Type: VdsGroupsAction group CREATE_HOST with role type ADMIN >2014-10-20 12:31:38,655 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-9) [3ab9577a] Running command: AddVdsSpmIdCommand internal: true. Entities affected : ID: 4ae674db-1cd0-4750-8934-40e94b0eea86 Type: VDS >2014-10-20 12:31:38,661 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-9) [3ab9577a] Lock freed to object EngineLock [exclusiveLocks= key: f8f5eaee-8fd0-4b45-87db-62d61b03a916 value: REGISTER_VDS >, sharedLocks= ] >2014-10-20 12:31:38,663 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-9) [3ab9577a] START, RemoveVdsVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = 4ae674db-1cd0-4750-8934-40e94b0eea86), log id: 34ffa2d2 >2014-10-20 12:31:38,663 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-9) [3ab9577a] FINISH, RemoveVdsVDSCommand, log id: 34ffa2d2 >2014-10-20 12:31:38,664 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-9) [3ab9577a] START, AddVdsVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = 4ae674db-1cd0-4750-8934-40e94b0eea86), log id: 785b5ac0 >2014-10-20 12:31:38,665 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-9) [3ab9577a] AddVds - entered , starting logic to add VDS 4ae674db-1cd0-4750-8934-40e94b0eea86 >2014-10-20 12:31:38,666 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-9) [3ab9577a] AddVds - VDS 4ae674db-1cd0-4750-8934-40e94b0eea86 was added, will try to add it to the resource manager >2014-10-20 12:31:38,666 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-9) [3ab9577a] Entered VdsManager constructor >2014-10-20 12:31:38,678 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-9) [3ab9577a] Initialize vdsBroker (alma07.qa.lab.tlv.redhat.com,54,321) >2014-10-20 12:31:38,679 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-9) [3ab9577a] VDS 4ae674db-1cd0-4750-8934-40e94b0eea86 was added to the Resource Manager >2014-10-20 12:31:38,679 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-9) [3ab9577a] FINISH, AddVdsVDSCommand, log id: 785b5ac0 >2014-10-20 12:31:38,687 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [3ab9577a] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host alma07.qa.lab.tlv.redhat.com. >2014-10-20 12:31:38,703 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-7-thread-11) [3fa09c86] Lock Acquired to object EngineLock [exclusiveLocks= key: 4ae674db-1cd0-4750-8934-40e94b0eea86 value: VDS >, sharedLocks= ] >2014-10-20 12:31:38,704 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [3ab9577a] Correlation ID: 2064bac5, Job ID: e050cc0c-a5a0-452c-99dd-5f3b38fddb1f, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com was added by admin. >2014-10-20 12:31:38,711 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-7-thread-11) [3fa09c86] Running command: InstallVdsInternalCommand internal: true. Entities affected : ID: 4ae674db-1cd0-4750-8934-40e94b0eea86 Type: VDS >2014-10-20 12:31:38,712 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-7-thread-11) [3fa09c86] Before Installation host 4ae674db-1cd0-4750-8934-40e94b0eea86, alma07.qa.lab.tlv.redhat.com >2014-10-20 12:31:38,716 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-11) [3fa09c86] START, SetVdsStatusVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = 4ae674db-1cd0-4750-8934-40e94b0eea86, status=Installing, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 40cd0546 >2014-10-20 12:31:38,719 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-11) [3fa09c86] FINISH, SetVdsStatusVDSCommand, log id: 40cd0546 >2014-10-20 12:31:38,830 INFO [org.ovirt.engine.core.bll.InstallerMessages] (org.ovirt.thread.pool-7-thread-11) [3fa09c86] Installation alma07.qa.lab.tlv.redhat.com: Connected to host alma07.qa.lab.tlv.redhat.com with SSH key fingerprint: b9:9d:07:b4:f0:87:c1:e7:36:1b:a6:71:2b:c2:51:21 >2014-10-20 12:31:38,931 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-11) [3fa09c86] Correlation ID: 3fa09c86, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Connected to host alma07.qa.lab.tlv.redhat.com with SSH key fingerprint: b9:9d:07:b4:f0:87:c1:e7:36:1b:a6:71:2b:c2:51:21. >2014-10-20 12:31:38,976 INFO [org.ovirt.engine.core.bll.VdsDeploy] (org.ovirt.thread.pool-7-thread-11) [3fa09c86] Installation of alma07.qa.lab.tlv.redhat.com. Executing command via SSH umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True < /var/cache/ovirt-engine/ovirt-host-deploy.tar >2014-10-20 12:31:38,990 INFO [org.ovirt.engine.core.uutils.ssh.SSHDialog] (org.ovirt.thread.pool-7-thread-11) SSH execute root@alma07.qa.lab.tlv.redhat.com 'umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True' >2014-10-20 12:31:39,961 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Stage: Initializing >2014-10-20 12:31:39,977 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3fa09c86, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Stage: Initializing. >2014-10-20 12:31:40,014 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Stage: Environment setup >2014-10-20 12:31:40,054 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3fa09c86, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Stage: Environment setup. >2014-10-20 12:31:40,054 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Stage: Environment packages setup >2014-10-20 12:31:40,060 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3fa09c86, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Stage: Environment packages setup. >2014-10-20 12:31:41,665 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Stage: Programs detection >2014-10-20 12:31:41,670 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3fa09c86, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Stage: Programs detection. >2014-10-20 12:31:41,670 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Stage: Environment customization >2014-10-20 12:31:41,675 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3fa09c86, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Stage: Environment customization. >2014-10-20 12:31:41,681 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-67) Initializing Host: alma07.qa.lab.tlv.redhat.com >2014-10-20 12:31:41,826 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Kdump supported >2014-10-20 12:31:41,832 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3fa09c86, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Kdump supported. >2014-10-20 12:31:41,841 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Logs at host located at: '/tmp/ovirt-host-deploy-20141020123839-y21oak.log' >2014-10-20 12:31:41,847 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3fa09c86, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Logs at host located at: '/tmp/ovirt-host-deploy-20141020123839-y21oak.log'. >2014-10-20 12:31:41,891 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Host alma07.qa.lab.tlv.redhat.com reports unique id 05A58EA4-E863-11E3-86FF-40F2E939C768 >2014-10-20 12:31:41,893 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Assigning unique id 05A58EA4-E863-11E3-86FF-40F2E939C768 to Host alma07.qa.lab.tlv.redhat.com >2014-10-20 12:31:41,973 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Disabling Kdump integration >2014-10-20 12:31:41,978 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3fa09c86, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Disabling Kdump integration. >2014-10-20 12:31:41,998 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Stage: Setup validation >2014-10-20 12:31:42,005 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3fa09c86, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Stage: Setup validation. >2014-10-20 12:31:42,052 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Hardware supports virtualization >2014-10-20 12:31:42,058 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3fa09c86, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Hardware supports virtualization. >2014-10-20 12:31:42,146 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Stage: Transaction setup >2014-10-20 12:31:42,151 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3fa09c86, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Stage: Transaction setup. >2014-10-20 12:31:42,254 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Stage: Misc configuration >2014-10-20 12:31:42,260 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3fa09c86, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Stage: Misc configuration. >2014-10-20 12:31:42,260 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Stage: Package installation >2014-10-20 12:31:42,265 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3fa09c86, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Stage: Package installation. >2014-10-20 12:31:49,768 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Yum Status: Downloading Packages >2014-10-20 12:31:49,774 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3fa09c86, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Yum Status: Downloading Packages. >2014-10-20 12:31:51,772 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Yum Download/Verify: 2:qemu-kvm-rhev-tools-0.12.1.2-2.448.el6.x86_64 >2014-10-20 12:31:51,778 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3fa09c86, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Yum Download/Verify: 2:qemu-kvm-rhev-tools-0.12.1.2-2.448.el6.x86_64. >2014-10-20 12:31:53,200 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Yum Download/Verify: tuned-0.2.19-13.el6.noarch >2014-10-20 12:31:53,207 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3fa09c86, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Yum Download/Verify: tuned-0.2.19-13.el6.noarch. >2014-10-20 12:31:53,207 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Yum Status: Check Package Signatures >2014-10-20 12:31:53,213 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3fa09c86, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Yum Status: Check Package Signatures. >2014-10-20 12:31:53,213 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Yum Status: Running Test Transaction >2014-10-20 12:31:53,218 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3fa09c86, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Yum Status: Running Test Transaction. >2014-10-20 12:31:53,310 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Yum Status: Running Transaction >2014-10-20 12:31:53,315 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3fa09c86, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Yum Status: Running Transaction. >2014-10-20 12:31:53,591 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Yum install: 1/2: 2:qemu-kvm-rhev-tools-0.12.1.2-2.448.el6.x86_64 >2014-10-20 12:31:53,599 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3fa09c86, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Yum install: 1/2: 2:qemu-kvm-rhev-tools-0.12.1.2-2.448.el6.x86_64. >2014-10-20 12:31:54,031 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Yum install: 2/2: tuned-0.2.19-13.el6.noarch >2014-10-20 12:31:54,037 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3fa09c86, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Yum install: 2/2: tuned-0.2.19-13.el6.noarch. >2014-10-20 12:31:55,348 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Yum Verify: 1/2: tuned.noarch 0:0.2.19-13.el6 - u >2014-10-20 12:31:55,353 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3fa09c86, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Yum Verify: 1/2: tuned.noarch 0:0.2.19-13.el6 - u. >2014-10-20 12:31:55,477 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Yum Verify: 2/2: qemu-kvm-rhev-tools.x86_64 2:0.12.1.2-2.448.el6 - u >2014-10-20 12:31:55,483 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3fa09c86, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Yum Verify: 2/2: qemu-kvm-rhev-tools.x86_64 2:0.12.1.2-2.448.el6 - u. >2014-10-20 12:31:55,700 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Stage: Misc configuration >2014-10-20 12:31:55,706 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3fa09c86, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Stage: Misc configuration. >2014-10-20 12:31:57,149 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Enrolling certificate >2014-10-20 12:31:57,154 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3fa09c86, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Enrolling certificate. >2014-10-20 12:31:58,587 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Stage: Transaction commit >2014-10-20 12:31:58,592 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3fa09c86, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Stage: Transaction commit. >2014-10-20 12:31:59,798 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Stage: Closing up >2014-10-20 12:31:59,805 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3fa09c86, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Stage: Closing up. >2014-10-20 12:32:01,002 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Stopping libvirtd >2014-10-20 12:32:01,008 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3fa09c86, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Stopping libvirtd. >2014-10-20 12:32:01,062 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Starting vdsm >2014-10-20 12:32:01,068 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3fa09c86, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Starting vdsm. >2014-10-20 12:32:03,479 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Stage: Pre-termination >2014-10-20 12:32:03,485 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3fa09c86, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Stage: Pre-termination. >2014-10-20 12:32:03,498 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20141020123203-alma07.qa.lab.tlv.redhat.com-3fa09c86.log' >2014-10-20 12:32:03,503 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3fa09c86, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20141020123203-alma07.qa.lab.tlv.redhat.com-3fa09c86.log'. >2014-10-20 12:32:03,720 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Stage: Termination >2014-10-20 12:32:03,725 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3fa09c86, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Stage: Termination. >2014-10-20 12:32:03,841 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-20 12:32:03,843 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:111) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-20 12:32:03,843 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:111) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-20 12:32:03,844 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (org.ovirt.thread.pool-7-thread-43) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:122) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$3.call(JsonRpcVdsServer.java:917) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$3.call(JsonRpcVdsServer.java:913) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$FutureCallable.call(JsonRpcVdsServer.java:405) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$FutureCallable.call(JsonRpcVdsServer.java:394) [vdsbroker.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_65] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_65] > at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_65] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:111) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-20 12:32:03,846 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-7-thread-11) [3fa09c86] java.util.concurrent.ExecutionException: org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcRunTimeException: Connection issues during send request >2014-10-20 12:32:03,847 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-7-thread-11) [3fa09c86] Command PollVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = 4ae674db-1cd0-4750-8934-40e94b0eea86) execution failed. Exception: RuntimeException: java.util.concurrent.ExecutionException: org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcRunTimeException: Connection issues during send request >2014-10-20 12:32:04,348 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-20 12:32:04,454 INFO [org.ovirt.engine.core.bll.network.NetworkConfigurator] (org.ovirt.thread.pool-7-thread-11) [3fa09c86] Engine managed to communicate with VDSM agent on host alma07.qa.lab.tlv.redhat.com >2014-10-20 12:32:04,912 INFO [org.ovirt.engine.core.bll.network.host.SetupNetworksCommand] (org.ovirt.thread.pool-7-thread-11) [793b8a8a] Running command: SetupNetworksCommand internal: true. Entities affected : ID: 4ae674db-1cd0-4750-8934-40e94b0eea86 Type: VDSAction group CONFIGURE_HOST_NETWORK with role type ADMIN >2014-10-20 12:32:04,914 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetupNetworksVDSCommand] (org.ovirt.thread.pool-7-thread-11) [793b8a8a] START, SetupNetworksVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = 4ae674db-1cd0-4750-8934-40e94b0eea86, force=false, checkConnectivity=true, conectivityTimeout=120, > networks=[rhevm {id=ef2adf9b-e3a8-47b9-ab30-075cc9cd0694, description=Management Network, comment=null, subnet=null, gateway=null, type=null, vlanId=null, stp=false, dataCenterId=f8f5eaee-8fd0-4b45-87db-62d61b03a916, mtu=0, vmNetwork=true, cluster=NetworkCluster {id={clusterId=null, networkId=null}, status=OPERATIONAL, display=true, required=true, migration=true}, providedBy=null, label=null, qosId=null}], > bonds=[], > interfaces=[eth3 {id=3f91cf9a-3d6e-4174-9f98-3dc5df068a82, vdsId=4ae674db-1cd0-4750-8934-40e94b0eea86, macAddress=40:f2:e9:39:c7:69, networkName=null, baseInterface=null, vlanId=null, bonded=null, bondName=null, bondOptions=null, bootProtocol=DHCP, address=, subnet=, gateway=null, mtu=1500, bridged=false, speed=0, type=0, networkImplementationDetails=null, qos=null qosOverridden=false, labels=null, customProperties=null}, > eth2 {id=7058a112-ca51-4e6c-a96f-a4e3308a8a6c, vdsId=4ae674db-1cd0-4750-8934-40e94b0eea86, macAddress=40:f2:e9:39:c7:68, networkName=null, baseInterface=null, vlanId=null, bonded=null, bondName=null, bondOptions=null, bootProtocol=DHCP, address=, subnet=, gateway=null, mtu=1500, bridged=false, speed=0, type=0, networkImplementationDetails=null, qos=null qosOverridden=false, labels=null, customProperties=null}, > eth1 {id=68378382-270e-4e5d-9105-0869d6f4ba91, vdsId=4ae674db-1cd0-4750-8934-40e94b0eea86, macAddress=90:e2:ba:7d:36:39, networkName=null, baseInterface=null, vlanId=null, bonded=null, bondName=null, bondOptions=null, bootProtocol=DHCP, address=, subnet=, gateway=null, mtu=1500, bridged=false, speed=0, type=0, networkImplementationDetails=null, qos=null qosOverridden=false, labels=null, customProperties=null}, > eth0 {id=4051da0b-b331-4954-8bd1-51af6cdb7f8a, vdsId=4ae674db-1cd0-4750-8934-40e94b0eea86, macAddress=90:e2:ba:7d:36:38, networkName=rhevm, baseInterface=null, vlanId=null, bonded=null, bondName=null, bondOptions=null, bootProtocol=DHCP, address=10.35.117.34, subnet=255.255.255.0, gateway=null, mtu=1500, bridged=false, speed=10000, type=0, networkImplementationDetails=null, qos=null qosOverridden=false, labels=null, customProperties=null}], > removedNetworks=[], > removedBonds=[]), log id: 5bba628 >2014-10-20 12:32:04,916 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetupNetworksVDSCommand] (org.ovirt.thread.pool-7-thread-11) [793b8a8a] FINISH, SetupNetworksVDSCommand, log id: 5bba628 >2014-10-20 12:32:04,919 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-20 12:32:13,808 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-20 12:32:13,944 INFO [org.ovirt.engine.core.bll.network.host.CommitNetworkChangesCommand] (org.ovirt.thread.pool-7-thread-11) [72b0280a] Running command: CommitNetworkChangesCommand internal: true. Entities affected : ID: 4ae674db-1cd0-4750-8934-40e94b0eea86 Type: VDSAction group CONFIGURE_HOST_NETWORK with role type ADMIN >2014-10-20 12:32:13,945 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetSafeNetworkConfigVDSCommand] (org.ovirt.thread.pool-7-thread-11) [72b0280a] START, SetSafeNetworkConfigVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = 4ae674db-1cd0-4750-8934-40e94b0eea86), log id: 5a7313c7 >2014-10-20 12:32:13,963 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetSafeNetworkConfigVDSCommand] (org.ovirt.thread.pool-7-thread-11) [72b0280a] FINISH, SetSafeNetworkConfigVDSCommand, log id: 5a7313c7 >2014-10-20 12:32:13,981 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-11) [72b0280a] Correlation ID: 72b0280a, Call Stack: null, Custom Event ID: -1, Message: Network changes were saved on host alma07.qa.lab.tlv.redhat.com >2014-10-20 12:32:13,983 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-11) [72b0280a] START, SetVdsStatusVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = 4ae674db-1cd0-4750-8934-40e94b0eea86, status=Initializing, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 55ce88aa >2014-10-20 12:32:13,988 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-11) [72b0280a] FINISH, SetVdsStatusVDSCommand, log id: 55ce88aa >2014-10-20 12:32:13,988 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-7-thread-11) [72b0280a] After Installation host alma07.qa.lab.tlv.redhat.com, VDS >2014-10-20 12:32:13,995 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-11) [72b0280a] Correlation ID: 3fa09c86, Job ID: e050cc0c-a5a0-452c-99dd-5f3b38fddb1f, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com installed >2014-10-20 12:32:14,019 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-7-thread-11) [72b0280a] Lock freed to object EngineLock [exclusiveLocks= key: 4ae674db-1cd0-4750-8934-40e94b0eea86 value: VDS >, sharedLocks= ] >2014-10-20 12:32:14,764 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-59) START, GetHardwareInfoVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = 4ae674db-1cd0-4750-8934-40e94b0eea86, vds=Host[alma07.qa.lab.tlv.redhat.com,4ae674db-1cd0-4750-8934-40e94b0eea86]), log id: 5caccf92 >2014-10-20 12:32:14,819 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-59) FINISH, GetHardwareInfoVDSCommand, log id: 5caccf92 >2014-10-20 12:32:14,855 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-59) [b9e0446] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 4ae674db-1cd0-4750-8934-40e94b0eea86 Type: VDS >2014-10-20 12:32:14,869 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-59) [2d9c5866] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 4ae674db-1cd0-4750-8934-40e94b0eea86 Type: VDS >2014-10-20 12:32:14,871 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-59) [2d9c5866] START, SetVdsStatusVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = 4ae674db-1cd0-4750-8934-40e94b0eea86, status=NonOperational, nonOperationalReason=ARCHITECTURE_INCOMPATIBLE_WITH_CLUSTER, stopSpmFailureLogged=false), log id: 20dd578 >2014-10-20 12:32:14,873 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-59) [2d9c5866] FINISH, SetVdsStatusVDSCommand, log id: 20dd578 >2014-10-20 12:32:14,885 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-59) [2d9c5866] Correlation ID: 2d9c5866, Job ID: 85e8aab1-5382-448b-894e-64bb3b3837f3, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com moved to Non-Operational state. >2014-10-20 12:32:14,892 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-59) [2d9c5866] Correlation ID: b9e0446, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has architecture x86_64 and cannot join Cluster cl_35_ppc which has architecture ppc64. >2014-10-20 12:32:14,937 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-59) [2d9c5866] START, GetHardwareInfoVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = 4ae674db-1cd0-4750-8934-40e94b0eea86, vds=Host[alma07.qa.lab.tlv.redhat.com,4ae674db-1cd0-4750-8934-40e94b0eea86]), log id: aa96a9a >2014-10-20 12:32:14,941 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-59) [2d9c5866] FINISH, GetHardwareInfoVDSCommand, log id: aa96a9a >2014-10-20 12:32:14,966 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-59) [64e411a1] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 4ae674db-1cd0-4750-8934-40e94b0eea86 Type: VDS >2014-10-20 12:32:14,983 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-59) [451a402b] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 4ae674db-1cd0-4750-8934-40e94b0eea86 Type: VDS >2014-10-20 12:32:14,985 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-59) [451a402b] START, SetVdsStatusVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = 4ae674db-1cd0-4750-8934-40e94b0eea86, status=NonOperational, nonOperationalReason=ARCHITECTURE_INCOMPATIBLE_WITH_CLUSTER, stopSpmFailureLogged=false), log id: 1efff370 >2014-10-20 12:32:14,987 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-59) [451a402b] FINISH, SetVdsStatusVDSCommand, log id: 1efff370 >2014-10-20 12:32:15,000 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-59) [451a402b] Correlation ID: 64e411a1, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has architecture x86_64 and cannot join Cluster cl_35_ppc which has architecture ppc64. >2014-10-20 12:32:15,006 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-59) [451a402b] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host alma07.qa.lab.tlv.redhat.com was set to NonOperational. >2014-10-20 12:32:15,029 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-59) [2f959b90] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 4ae674db-1cd0-4750-8934-40e94b0eea86 Type: VDS >2014-10-20 12:32:15,043 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-59) [e9235bd] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 4ae674db-1cd0-4750-8934-40e94b0eea86 Type: VDS >2014-10-20 12:32:15,046 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-59) [e9235bd] START, SetVdsStatusVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = 4ae674db-1cd0-4750-8934-40e94b0eea86, status=NonOperational, nonOperationalReason=ARCHITECTURE_INCOMPATIBLE_WITH_CLUSTER, stopSpmFailureLogged=false), log id: 1debf7b9 >2014-10-20 12:32:15,050 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-59) [e9235bd] FINISH, SetVdsStatusVDSCommand, log id: 1debf7b9 >2014-10-20 12:32:15,062 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-59) [e9235bd] Correlation ID: 2f959b90, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has architecture x86_64 and cannot join Cluster cl_35_ppc which has architecture ppc64. >2014-10-20 12:32:15,065 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-59) [4f1e1d71] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 4ae674db-1cd0-4750-8934-40e94b0eea86 Type: VDS >2014-10-20 12:32:15,067 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-59) [4f1e1d71] Host 4ae674db-1cd0-4750-8934-40e94b0eea86 : alma07.qa.lab.tlv.redhat.com is already in NonOperational status for reason ARCHITECTURE_INCOMPATIBLE_WITH_CLUSTER. SetNonOperationalVds command is skipped. >2014-10-20 12:33:54,547 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcherImpl.read0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) [rt.jar:1.7.0_65] > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) [rt.jar:1.7.0_65] > at sun.nio.ch.IOUtil.read(IOUtil.java:197) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.read(SSLEngineNioHelper.java:51) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.read(SSLClient.java:81) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient.processIncoming(StompCommonClient.java:90) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.process(ReactorClient.java:168) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:115) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] > >2014-10-20 12:33:59,167 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (ajp-/127.0.0.1:8702-6) [767d2a49] Lock Acquired to object EngineLock [exclusiveLocks= key: 4ae674db-1cd0-4750-8934-40e94b0eea86 value: VDS >, sharedLocks= ] >2014-10-20 12:33:59,208 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-7-thread-24) [767d2a49] Running command: ActivateVdsCommand internal: false. Entities affected : ID: 4ae674db-1cd0-4750-8934-40e94b0eea86 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-10-20 12:33:59,208 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-7-thread-24) [767d2a49] Before acquiring lock in order to prevent monitoring for host alma07.qa.lab.tlv.redhat.com from data-center dc_35 >2014-10-20 12:33:59,208 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-7-thread-24) [767d2a49] Lock acquired, from now a monitoring of host will be skipped for host alma07.qa.lab.tlv.redhat.com from data-center dc_35 >2014-10-20 12:33:59,217 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-24) [767d2a49] START, SetVdsStatusVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = 4ae674db-1cd0-4750-8934-40e94b0eea86, status=Unassigned, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 6a2de28f >2014-10-20 12:33:59,220 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-24) [767d2a49] FINISH, SetVdsStatusVDSCommand, log id: 6a2de28f >2014-10-20 12:33:59,227 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-7-thread-24) [767d2a49] Activate finished. Lock released. Monitoring can run now for host alma07.qa.lab.tlv.redhat.com from data-center dc_35 >2014-10-20 12:33:59,231 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-24) [767d2a49] Correlation ID: 767d2a49, Job ID: 34804e7f-30b1-4b8e-a9ee-4438184ba352, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com was activated by admin. >2014-10-20 12:33:59,234 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-7-thread-24) [767d2a49] Lock freed to object EngineLock [exclusiveLocks= key: 4ae674db-1cd0-4750-8934-40e94b0eea86 value: VDS >, sharedLocks= ] >2014-10-20 12:34:00,177 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-20 12:34:00,571 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-82) START, GetHardwareInfoVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = 4ae674db-1cd0-4750-8934-40e94b0eea86, vds=Host[alma07.qa.lab.tlv.redhat.com,4ae674db-1cd0-4750-8934-40e94b0eea86]), log id: 720d8ee9 >2014-10-20 12:34:00,583 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-82) FINISH, GetHardwareInfoVDSCommand, log id: 720d8ee9 >2014-10-20 12:34:00,620 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-82) [601f7dcc] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 4ae674db-1cd0-4750-8934-40e94b0eea86 Type: VDS >2014-10-20 12:34:00,622 ERROR [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-82) [601f7dcc] Could not find server cpu for server 4ae674db-1cd0-4750-8934-40e94b0eea86:alma07.qa.lab.tlv.redhat.com, flags: powernv,model_POWER7_v2.3 >2014-10-20 12:34:00,633 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-82) [53daf285] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 4ae674db-1cd0-4750-8934-40e94b0eea86 Type: VDS >2014-10-20 12:34:00,640 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-82) [53daf285] START, SetVdsStatusVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = 4ae674db-1cd0-4750-8934-40e94b0eea86, status=NonOperational, nonOperationalReason=CPU_TYPE_INCOMPATIBLE_WITH_CLUSTER, stopSpmFailureLogged=false), log id: 6d449e8e >2014-10-20 12:34:00,643 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-82) [53daf285] FINISH, SetVdsStatusVDSCommand, log id: 6d449e8e >2014-10-20 12:34:00,655 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-82) [53daf285] Correlation ID: 53daf285, Job ID: d459f834-16c4-4a86-ac9a-5b22a23084e9, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com moved to Non-Operational state. >2014-10-20 12:34:00,696 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-82) [53daf285] Correlation ID: 601f7dcc, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com moved to Non-Operational state as host CPU type is not supported in this cluster compatibility version or is not supported at all >2014-10-20 12:34:00,709 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-82) [53daf285] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host alma07.qa.lab.tlv.redhat.com was set to NonOperational. >2014-10-20 12:34:00,728 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-82) [6edeb7d0] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 4ae674db-1cd0-4750-8934-40e94b0eea86 Type: VDS >2014-10-20 12:34:00,730 ERROR [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-82) [6edeb7d0] Could not find server cpu for server 4ae674db-1cd0-4750-8934-40e94b0eea86:alma07.qa.lab.tlv.redhat.com, flags: powernv,model_POWER7_v2.3 >2014-10-20 12:34:00,744 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-82) [1227b88b] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 4ae674db-1cd0-4750-8934-40e94b0eea86 Type: VDS >2014-10-20 12:34:00,746 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-82) [1227b88b] START, SetVdsStatusVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = 4ae674db-1cd0-4750-8934-40e94b0eea86, status=NonOperational, nonOperationalReason=CPU_TYPE_INCOMPATIBLE_WITH_CLUSTER, stopSpmFailureLogged=false), log id: 2f71becb >2014-10-20 12:34:00,750 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-82) [1227b88b] FINISH, SetVdsStatusVDSCommand, log id: 2f71becb >2014-10-20 12:34:00,763 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-82) [1227b88b] Correlation ID: 6edeb7d0, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com moved to Non-Operational state as host CPU type is not supported in this cluster compatibility version or is not supported at all >2014-10-20 12:34:00,767 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-82) [682273c3] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 4ae674db-1cd0-4750-8934-40e94b0eea86 Type: VDS >2014-10-20 12:34:00,768 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-82) [682273c3] Host 4ae674db-1cd0-4750-8934-40e94b0eea86 : alma07.qa.lab.tlv.redhat.com is already in NonOperational status for reason CPU_TYPE_INCOMPATIBLE_WITH_CLUSTER. SetNonOperationalVds command is skipped. >2014-10-20 12:38:16,302 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-83) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 12:39:21,175 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-7-thread-25) [3fea960c] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: 4ae674db-1cd0-4750-8934-40e94b0eea86 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-10-20 12:39:21,178 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-25) [3fea960c] START, SetVdsStatusVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = 4ae674db-1cd0-4750-8934-40e94b0eea86, status=PreparingForMaintenance, nonOperationalReason=NONE, stopSpmFailureLogged=true), log id: 286dcbcf >2014-10-20 12:39:21,181 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-25) [3fea960c] FINISH, SetVdsStatusVDSCommand, log id: 286dcbcf >2014-10-20 12:39:21,260 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (org.ovirt.thread.pool-7-thread-25) [3fea960c] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: 4ae674db-1cd0-4750-8934-40e94b0eea86 Type: VDS >2014-10-20 12:39:21,266 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-25) [3fea960c] Correlation ID: 3fea960c, Job ID: 53cdd9bf-562f-4c19-9a6d-03d3ae9e5450, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com was switched to Maintenance mode by admin. >2014-10-20 12:39:22,168 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-38) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 12:39:22,174 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-38) Updated vds status from Preparing for Maintenance to Maintenance in database, vds = 4ae674db-1cd0-4750-8934-40e94b0eea86 : alma07.qa.lab.tlv.redhat.com >2014-10-20 12:39:22,194 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-7-thread-24) Clearing cache of pool: f8f5eaee-8fd0-4b45-87db-62d61b03a916 for problematic entities of VDS: alma07.qa.lab.tlv.redhat.com. >2014-10-20 12:39:22,194 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-7-thread-24) Removing vds [4ae674db-1cd0-4750-8934-40e94b0eea86] from the domain in maintenance cache >2014-10-20 12:39:22,194 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-7-thread-24) Removing host(s) [4ae674db-1cd0-4750-8934-40e94b0eea86] from hosts unseen domain report cache >2014-10-20 12:39:22,197 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-38) START, DisconnectStoragePoolVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = 4ae674db-1cd0-4750-8934-40e94b0eea86, storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, vds_spm_id = 3), log id: cfc3468 >2014-10-20 12:39:22,201 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-38) FINISH, DisconnectStoragePoolVDSCommand, log id: cfc3468 >2014-10-20 12:39:22,202 INFO [org.ovirt.engine.core.bll.storage.DisconnectHostFromStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-38) [17521ead] Running command: DisconnectHostFromStoragePoolServersCommand internal: true. Entities affected : ID: f8f5eaee-8fd0-4b45-87db-62d61b03a916 Type: StoragePool >2014-10-20 12:39:22,209 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-38) [17521ead] START, DisconnectStorageServerVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = 4ae674db-1cd0-4750-8934-40e94b0eea86, storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, storageType = NFS, connectionList = [{ id: 5eec92db-0201-4aa5-8598-1fe041c7a057, connection: 10.35.160.108:/RHEV/artyom_storage_rhevm_35, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 19ec569f >2014-10-20 12:39:22,360 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-38) [17521ead] FINISH, DisconnectStorageServerVDSCommand, return: {5eec92db-0201-4aa5-8598-1fe041c7a057=477}, log id: 19ec569f >2014-10-20 12:59:23,994 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-4) Deactivating non existing principal admin::internal >2014-10-20 12:59:33,320 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-78) Attempting to update VMs/Templates Ovf. >2014-10-20 13:12:10,825 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-23) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 13:38:19,744 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-77) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 13:59:33,333 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-13) Attempting to update VMs/Templates Ovf. >2014-10-20 14:12:15,023 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-79) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 418 seconds while maximum configured value is 300 seconds. >2014-10-20 14:18:12,520 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (ajp-/127.0.0.1:8702-10) [1808e532] Lock Acquired to object EngineLock [exclusiveLocks= key: 4ae674db-1cd0-4750-8934-40e94b0eea86 value: VDS >, sharedLocks= ] >2014-10-20 14:18:12,540 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-7-thread-12) [1808e532] Running command: ActivateVdsCommand internal: false. Entities affected : ID: 4ae674db-1cd0-4750-8934-40e94b0eea86 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-10-20 14:18:12,540 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-7-thread-12) [1808e532] Before acquiring lock in order to prevent monitoring for host alma07.qa.lab.tlv.redhat.com from data-center dc_35 >2014-10-20 14:18:12,540 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-7-thread-12) [1808e532] Lock acquired, from now a monitoring of host will be skipped for host alma07.qa.lab.tlv.redhat.com from data-center dc_35 >2014-10-20 14:18:12,547 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-12) [1808e532] START, SetVdsStatusVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = 4ae674db-1cd0-4750-8934-40e94b0eea86, status=Unassigned, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 5214f3fd >2014-10-20 14:18:12,551 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-12) [1808e532] FINISH, SetVdsStatusVDSCommand, log id: 5214f3fd >2014-10-20 14:18:12,557 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-7-thread-12) [1808e532] Activate finished. Lock released. Monitoring can run now for host alma07.qa.lab.tlv.redhat.com from data-center dc_35 >2014-10-20 14:18:12,562 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-12) [1808e532] Correlation ID: 1808e532, Job ID: 40e2521b-e38a-4ff6-a81d-5d940d34194e, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com was activated by admin. >2014-10-20 14:18:12,579 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-7-thread-12) [1808e532] Lock freed to object EngineLock [exclusiveLocks= key: 4ae674db-1cd0-4750-8934-40e94b0eea86 value: VDS >, sharedLocks= ] >2014-10-20 14:18:13,676 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-20 14:18:14,163 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-65) START, GetHardwareInfoVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = 4ae674db-1cd0-4750-8934-40e94b0eea86, vds=Host[alma07.qa.lab.tlv.redhat.com,4ae674db-1cd0-4750-8934-40e94b0eea86]), log id: 53fd918f >2014-10-20 14:18:14,170 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-65) FINISH, GetHardwareInfoVDSCommand, log id: 53fd918f >2014-10-20 14:18:14,192 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-65) [2777e306] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 4ae674db-1cd0-4750-8934-40e94b0eea86 Type: VDS >2014-10-20 14:18:14,231 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-65) [2668369b] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 4ae674db-1cd0-4750-8934-40e94b0eea86 Type: VDS >2014-10-20 14:18:14,233 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-65) [2668369b] START, SetVdsStatusVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = 4ae674db-1cd0-4750-8934-40e94b0eea86, status=NonOperational, nonOperationalReason=EMULATED_MACHINES_INCOMPATIBLE_WITH_CLUSTER, stopSpmFailureLogged=false), log id: 7044d576 >2014-10-20 14:18:14,237 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-65) [2668369b] FINISH, SetVdsStatusVDSCommand, log id: 7044d576 >2014-10-20 14:18:14,255 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-65) [2668369b] Correlation ID: 2668369b, Job ID: 635ef0b1-8697-48c4-bd10-f5f3d0998ba8, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com does not comply with the cluster cl_35_ppc emulated machines. The Hosts emulated machines are <UNKNOWN> and the cluster is rhel6.5.0} >2014-10-20 14:18:14,264 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-65) [2668369b] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host alma07.qa.lab.tlv.redhat.com was set to NonOperational. >2014-10-20 14:18:14,282 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-65) [2ecca2af] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 4ae674db-1cd0-4750-8934-40e94b0eea86 Type: VDS >2014-10-20 14:18:14,291 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-65) [325f84a] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 4ae674db-1cd0-4750-8934-40e94b0eea86 Type: VDS >2014-10-20 14:18:14,292 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-65) [325f84a] Host 4ae674db-1cd0-4750-8934-40e94b0eea86 : alma07.qa.lab.tlv.redhat.com is already in NonOperational status for reason EMULATED_MACHINES_INCOMPATIBLE_WITH_CLUSTER. SetNonOperationalVds command is skipped. >2014-10-20 14:18:39,328 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-7-thread-48) [491ed496] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: 4ae674db-1cd0-4750-8934-40e94b0eea86 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-10-20 14:18:39,330 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-48) [491ed496] START, SetVdsStatusVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = 4ae674db-1cd0-4750-8934-40e94b0eea86, status=PreparingForMaintenance, nonOperationalReason=NONE, stopSpmFailureLogged=true), log id: 157cdb41 >2014-10-20 14:18:39,334 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-48) [491ed496] FINISH, SetVdsStatusVDSCommand, log id: 157cdb41 >2014-10-20 14:18:39,433 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (org.ovirt.thread.pool-7-thread-48) [491ed496] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: 4ae674db-1cd0-4750-8934-40e94b0eea86 Type: VDS >2014-10-20 14:18:39,440 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-48) [491ed496] Correlation ID: 491ed496, Job ID: 487afcd4-b32a-45d0-9781-9d91e12a070c, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com was switched to Maintenance mode by admin. >2014-10-20 14:18:41,339 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 14:18:41,345 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-20) Updated vds status from Preparing for Maintenance to Maintenance in database, vds = 4ae674db-1cd0-4750-8934-40e94b0eea86 : alma07.qa.lab.tlv.redhat.com >2014-10-20 14:18:41,361 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-7-thread-40) Clearing cache of pool: f8f5eaee-8fd0-4b45-87db-62d61b03a916 for problematic entities of VDS: alma07.qa.lab.tlv.redhat.com. >2014-10-20 14:18:41,361 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-7-thread-40) Removing vds [4ae674db-1cd0-4750-8934-40e94b0eea86] from the domain in maintenance cache >2014-10-20 14:18:41,361 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-7-thread-40) Removing host(s) [4ae674db-1cd0-4750-8934-40e94b0eea86] from hosts unseen domain report cache >2014-10-20 14:18:41,364 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-20) START, DisconnectStoragePoolVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = 4ae674db-1cd0-4750-8934-40e94b0eea86, storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, vds_spm_id = 3), log id: 42a988e4 >2014-10-20 14:18:41,369 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-20) FINISH, DisconnectStoragePoolVDSCommand, log id: 42a988e4 >2014-10-20 14:18:41,370 INFO [org.ovirt.engine.core.bll.storage.DisconnectHostFromStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-20) [721a74f9] Running command: DisconnectHostFromStoragePoolServersCommand internal: true. Entities affected : ID: f8f5eaee-8fd0-4b45-87db-62d61b03a916 Type: StoragePool >2014-10-20 14:18:41,381 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-20) [721a74f9] START, DisconnectStorageServerVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = 4ae674db-1cd0-4750-8934-40e94b0eea86, storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, storageType = NFS, connectionList = [{ id: 5eec92db-0201-4aa5-8598-1fe041c7a057, connection: 10.35.160.108:/RHEV/artyom_storage_rhevm_35, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 1f80c54d >2014-10-20 14:18:41,511 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-20) [721a74f9] FINISH, DisconnectStorageServerVDSCommand, return: {5eec92db-0201-4aa5-8598-1fe041c7a057=477}, log id: 1f80c54d >2014-10-20 14:18:47,162 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (ajp-/127.0.0.1:8702-6) [442f13f9] Lock Acquired to object EngineLock [exclusiveLocks= key: 4ae674db-1cd0-4750-8934-40e94b0eea86 value: VDS >, sharedLocks= ] >2014-10-20 14:18:47,170 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (org.ovirt.thread.pool-7-thread-36) [442f13f9] Running command: RemoveVdsCommand internal: false. Entities affected : ID: 4ae674db-1cd0-4750-8934-40e94b0eea86 Type: VDSAction group DELETE_HOST with role type ADMIN >2014-10-20 14:18:47,211 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (org.ovirt.thread.pool-7-thread-36) [442f13f9] START, RemoveVdsVDSCommand( HostId = 4ae674db-1cd0-4750-8934-40e94b0eea86), log id: 25522c95 >2014-10-20 14:18:47,212 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (org.ovirt.thread.pool-7-thread-36) [442f13f9] vdsManager::disposing >2014-10-20 14:18:47,212 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (org.ovirt.thread.pool-7-thread-36) [442f13f9] FINISH, RemoveVdsVDSCommand, log id: 25522c95 >2014-10-20 14:18:47,215 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-36) [442f13f9] Correlation ID: 442f13f9, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com was removed by admin. >2014-10-20 14:18:47,215 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (org.ovirt.thread.pool-7-thread-36) [442f13f9] Lock freed to object EngineLock [exclusiveLocks= key: 4ae674db-1cd0-4750-8934-40e94b0eea86 value: VDS >, sharedLocks= ] >2014-10-20 14:19:08,001 INFO [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-6) [7a73ca5f] Running command: AddVdsCommand internal: false. Entities affected : ID: 63702093-c04f-417c-8045-aa56334b54ee Type: VdsGroupsAction group CREATE_HOST with role type ADMIN >2014-10-20 14:19:08,019 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-6) [145d785d] Running command: AddVdsSpmIdCommand internal: true. Entities affected : ID: d5b2492d-8110-4614-9ce4-4a7a118c7942 Type: VDS >2014-10-20 14:19:08,025 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-6) [145d785d] Lock freed to object EngineLock [exclusiveLocks= key: f8f5eaee-8fd0-4b45-87db-62d61b03a916 value: REGISTER_VDS >, sharedLocks= ] >2014-10-20 14:19:08,027 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-6) [145d785d] START, RemoveVdsVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942), log id: 272589c8 >2014-10-20 14:19:08,027 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-6) [145d785d] FINISH, RemoveVdsVDSCommand, log id: 272589c8 >2014-10-20 14:19:08,028 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-6) [145d785d] START, AddVdsVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942), log id: 7afa5c0b >2014-10-20 14:19:08,028 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-6) [145d785d] AddVds - entered , starting logic to add VDS d5b2492d-8110-4614-9ce4-4a7a118c7942 >2014-10-20 14:19:08,030 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-6) [145d785d] AddVds - VDS d5b2492d-8110-4614-9ce4-4a7a118c7942 was added, will try to add it to the resource manager >2014-10-20 14:19:08,030 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-6) [145d785d] Entered VdsManager constructor >2014-10-20 14:19:08,042 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-6) [145d785d] Initialize vdsBroker (alma07.qa.lab.tlv.redhat.com,54,321) >2014-10-20 14:19:08,043 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-6) [145d785d] VDS d5b2492d-8110-4614-9ce4-4a7a118c7942 was added to the Resource Manager >2014-10-20 14:19:08,043 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-6) [145d785d] FINISH, AddVdsVDSCommand, log id: 7afa5c0b >2014-10-20 14:19:08,052 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) [145d785d] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host alma07.qa.lab.tlv.redhat.com. >2014-10-20 14:19:08,067 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-7-thread-37) [6826e7cc] Lock Acquired to object EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS >, sharedLocks= ] >2014-10-20 14:19:08,070 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) [145d785d] Correlation ID: 7a73ca5f, Job ID: 108771a0-6a0e-4773-bfd2-09ed11b5bd94, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com was added by admin. >2014-10-20 14:19:08,078 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-7-thread-37) [6826e7cc] Running command: InstallVdsInternalCommand internal: true. Entities affected : ID: d5b2492d-8110-4614-9ce4-4a7a118c7942 Type: VDS >2014-10-20 14:19:08,078 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-7-thread-37) [6826e7cc] Before Installation host d5b2492d-8110-4614-9ce4-4a7a118c7942, alma07.qa.lab.tlv.redhat.com >2014-10-20 14:19:08,082 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-37) [6826e7cc] START, SetVdsStatusVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, status=Installing, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: b286d3d >2014-10-20 14:19:08,085 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-37) [6826e7cc] FINISH, SetVdsStatusVDSCommand, log id: b286d3d >2014-10-20 14:19:08,130 INFO [org.ovirt.engine.core.bll.InstallerMessages] (org.ovirt.thread.pool-7-thread-37) [6826e7cc] Installation alma07.qa.lab.tlv.redhat.com: Connected to host alma07.qa.lab.tlv.redhat.com with SSH key fingerprint: b9:9d:07:b4:f0:87:c1:e7:36:1b:a6:71:2b:c2:51:21 >2014-10-20 14:19:08,136 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-37) [6826e7cc] Correlation ID: 6826e7cc, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Connected to host alma07.qa.lab.tlv.redhat.com with SSH key fingerprint: b9:9d:07:b4:f0:87:c1:e7:36:1b:a6:71:2b:c2:51:21. >2014-10-20 14:19:08,208 INFO [org.ovirt.engine.core.bll.VdsDeploy] (org.ovirt.thread.pool-7-thread-37) [6826e7cc] Installation of alma07.qa.lab.tlv.redhat.com. Executing command via SSH umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True < /var/cache/ovirt-engine/ovirt-host-deploy.tar >2014-10-20 14:19:08,222 INFO [org.ovirt.engine.core.uutils.ssh.SSHDialog] (org.ovirt.thread.pool-7-thread-37) SSH execute root@alma07.qa.lab.tlv.redhat.com 'umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True' >2014-10-20 14:19:09,037 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Stage: Initializing >2014-10-20 14:19:09,043 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6826e7cc, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Stage: Initializing. >2014-10-20 14:19:09,060 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Stage: Environment setup >2014-10-20 14:19:09,065 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6826e7cc, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Stage: Environment setup. >2014-10-20 14:19:09,070 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Stage: Environment packages setup >2014-10-20 14:19:09,074 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6826e7cc, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Stage: Environment packages setup. >2014-10-20 14:19:10,625 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Stage: Programs detection >2014-10-20 14:19:10,630 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6826e7cc, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Stage: Programs detection. >2014-10-20 14:19:10,630 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Stage: Environment customization >2014-10-20 14:19:10,635 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6826e7cc, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Stage: Environment customization. >2014-10-20 14:19:10,765 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Kdump supported >2014-10-20 14:19:10,770 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6826e7cc, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Kdump supported. >2014-10-20 14:19:10,782 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Logs at host located at: '/tmp/ovirt-host-deploy-20141020142608-5izn5p.log' >2014-10-20 14:19:10,786 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6826e7cc, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Logs at host located at: '/tmp/ovirt-host-deploy-20141020142608-5izn5p.log'. >2014-10-20 14:19:10,826 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Host alma07.qa.lab.tlv.redhat.com reports unique id 05A58EA4-E863-11E3-86FF-40F2E939C768 >2014-10-20 14:19:10,828 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Assigning unique id 05A58EA4-E863-11E3-86FF-40F2E939C768 to Host alma07.qa.lab.tlv.redhat.com >2014-10-20 14:19:11,000 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Disabling Kdump integration >2014-10-20 14:19:11,009 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6826e7cc, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Disabling Kdump integration. >2014-10-20 14:19:11,021 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Stage: Setup validation >2014-10-20 14:19:11,026 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6826e7cc, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Stage: Setup validation. >2014-10-20 14:19:11,046 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-23) Initializing Host: alma07.qa.lab.tlv.redhat.com >2014-10-20 14:19:11,064 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Hardware supports virtualization >2014-10-20 14:19:11,070 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6826e7cc, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Hardware supports virtualization. >2014-10-20 14:19:11,158 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Stage: Transaction setup >2014-10-20 14:19:11,163 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6826e7cc, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Stage: Transaction setup. >2014-10-20 14:19:11,269 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Stage: Misc configuration >2014-10-20 14:19:11,282 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6826e7cc, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Stage: Misc configuration. >2014-10-20 14:19:11,282 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Stage: Package installation >2014-10-20 14:19:11,287 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6826e7cc, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Stage: Package installation. >2014-10-20 14:19:20,416 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Stage: Misc configuration >2014-10-20 14:19:20,423 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6826e7cc, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Stage: Misc configuration. >2014-10-20 14:19:20,424 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Setting time >2014-10-20 14:19:20,429 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6826e7cc, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Setting time. >2014-10-20 14:19:23,048 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Enrolling certificate >2014-10-20 14:19:23,054 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6826e7cc, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Enrolling certificate. >2014-10-20 14:19:24,465 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Stage: Transaction commit >2014-10-20 14:19:24,471 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6826e7cc, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Stage: Transaction commit. >2014-10-20 14:19:25,062 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Stage: Closing up >2014-10-20 14:19:25,068 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6826e7cc, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Stage: Closing up. >2014-10-20 14:19:26,194 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Stopping libvirtd >2014-10-20 14:19:26,199 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6826e7cc, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Stopping libvirtd. >2014-10-20 14:19:26,258 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Starting vdsm >2014-10-20 14:19:26,263 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6826e7cc, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Starting vdsm. >2014-10-20 14:19:28,620 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Stage: Pre-termination >2014-10-20 14:19:28,626 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6826e7cc, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Stage: Pre-termination. >2014-10-20 14:19:28,633 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20141020141928-alma07.qa.lab.tlv.redhat.com-6826e7cc.log' >2014-10-20 14:19:28,645 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6826e7cc, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20141020141928-alma07.qa.lab.tlv.redhat.com-6826e7cc.log'. >2014-10-20 14:19:28,868 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation alma07.qa.lab.tlv.redhat.com: Stage: Termination >2014-10-20 14:19:28,874 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6826e7cc, Call Stack: null, Custom Event ID: -1, Message: Installing Host alma07.qa.lab.tlv.redhat.com. Stage: Termination. >2014-10-20 14:19:28,998 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-20 14:19:28,999 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:111) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-20 14:19:28,999 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:111) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-20 14:19:28,999 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (org.ovirt.thread.pool-7-thread-32) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:122) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$3.call(JsonRpcVdsServer.java:917) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$3.call(JsonRpcVdsServer.java:913) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$FutureCallable.call(JsonRpcVdsServer.java:405) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$FutureCallable.call(JsonRpcVdsServer.java:394) [vdsbroker.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_65] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_65] > at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_65] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:111) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-20 14:19:29,001 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-7-thread-37) [6826e7cc] java.util.concurrent.ExecutionException: org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcRunTimeException: Connection issues during send request >2014-10-20 14:19:29,004 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-7-thread-37) [6826e7cc] Command PollVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942) execution failed. Exception: RuntimeException: java.util.concurrent.ExecutionException: org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcRunTimeException: Connection issues during send request >2014-10-20 14:19:29,505 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-20 14:19:29,624 INFO [org.ovirt.engine.core.bll.network.NetworkConfigurator] (org.ovirt.thread.pool-7-thread-37) [6826e7cc] Engine managed to communicate with VDSM agent on host alma07.qa.lab.tlv.redhat.com >2014-10-20 14:19:30,005 INFO [org.ovirt.engine.core.bll.network.NetworkConfigurator] (org.ovirt.thread.pool-7-thread-37) [6826e7cc] The management network rhevm is already configured on host alma07.qa.lab.tlv.redhat.com >2014-10-20 14:19:30,007 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-37) [6826e7cc] START, SetVdsStatusVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, status=Initializing, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 547d60e4 >2014-10-20 14:19:30,010 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-37) [6826e7cc] FINISH, SetVdsStatusVDSCommand, log id: 547d60e4 >2014-10-20 14:19:30,010 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-7-thread-37) [6826e7cc] After Installation host alma07.qa.lab.tlv.redhat.com, VDS >2014-10-20 14:19:30,031 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-37) [6826e7cc] Correlation ID: 6826e7cc, Job ID: 108771a0-6a0e-4773-bfd2-09ed11b5bd94, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com installed >2014-10-20 14:19:30,034 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-7-thread-37) [6826e7cc] Lock freed to object EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS >, sharedLocks= ] >2014-10-20 14:19:32,106 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-11) [43a5b796] START, GetHardwareInfoVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]), log id: 64d2aea1 >2014-10-20 14:19:32,133 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-11) [43a5b796] FINISH, GetHardwareInfoVDSCommand, log id: 64d2aea1 >2014-10-20 14:19:32,160 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-11) [3a03bed1] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: d5b2492d-8110-4614-9ce4-4a7a118c7942 Type: VDS >2014-10-20 14:19:32,175 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-11) [29d0a02d] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: d5b2492d-8110-4614-9ce4-4a7a118c7942 Type: VDS >2014-10-20 14:19:32,177 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-11) [29d0a02d] START, SetVdsStatusVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, status=NonOperational, nonOperationalReason=ARCHITECTURE_INCOMPATIBLE_WITH_CLUSTER, stopSpmFailureLogged=false), log id: 7ec220f5 >2014-10-20 14:19:32,180 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-11) [29d0a02d] FINISH, SetVdsStatusVDSCommand, log id: 7ec220f5 >2014-10-20 14:19:32,191 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) [29d0a02d] Correlation ID: 29d0a02d, Job ID: 7b4cdea2-f253-46a4-ae33-1eda07748c2a, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com moved to Non-Operational state. >2014-10-20 14:19:32,198 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) [29d0a02d] Correlation ID: 3a03bed1, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has architecture x86_64 and cannot join Cluster cl_35_ppc which has architecture ppc64. >2014-10-20 14:19:32,238 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-11) [29d0a02d] START, GetHardwareInfoVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]), log id: 1ef52a77 >2014-10-20 14:19:32,242 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-11) [29d0a02d] FINISH, GetHardwareInfoVDSCommand, log id: 1ef52a77 >2014-10-20 14:19:32,268 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-11) [51fc356c] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: d5b2492d-8110-4614-9ce4-4a7a118c7942 Type: VDS >2014-10-20 14:19:32,281 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-11) [29665cf9] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: d5b2492d-8110-4614-9ce4-4a7a118c7942 Type: VDS >2014-10-20 14:19:32,283 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-11) [29665cf9] START, SetVdsStatusVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, status=NonOperational, nonOperationalReason=ARCHITECTURE_INCOMPATIBLE_WITH_CLUSTER, stopSpmFailureLogged=false), log id: 13190cd2 >2014-10-20 14:19:32,286 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-11) [29665cf9] FINISH, SetVdsStatusVDSCommand, log id: 13190cd2 >2014-10-20 14:19:32,298 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) [29665cf9] Correlation ID: 51fc356c, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has architecture x86_64 and cannot join Cluster cl_35_ppc which has architecture ppc64. >2014-10-20 14:19:32,305 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) [29665cf9] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host alma07.qa.lab.tlv.redhat.com was set to NonOperational. >2014-10-20 14:19:32,324 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-11) [66b11343] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: d5b2492d-8110-4614-9ce4-4a7a118c7942 Type: VDS >2014-10-20 14:19:32,337 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-11) [553b2cf3] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: d5b2492d-8110-4614-9ce4-4a7a118c7942 Type: VDS >2014-10-20 14:19:32,339 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-11) [553b2cf3] START, SetVdsStatusVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, status=NonOperational, nonOperationalReason=ARCHITECTURE_INCOMPATIBLE_WITH_CLUSTER, stopSpmFailureLogged=false), log id: 78ad9321 >2014-10-20 14:19:32,343 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-11) [553b2cf3] FINISH, SetVdsStatusVDSCommand, log id: 78ad9321 >2014-10-20 14:19:32,355 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) [553b2cf3] Correlation ID: 66b11343, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has architecture x86_64 and cannot join Cluster cl_35_ppc which has architecture ppc64. >2014-10-20 14:19:32,358 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-11) [5f02ebd7] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: d5b2492d-8110-4614-9ce4-4a7a118c7942 Type: VDS >2014-10-20 14:19:32,360 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-11) [5f02ebd7] Host d5b2492d-8110-4614-9ce4-4a7a118c7942 : alma07.qa.lab.tlv.redhat.com is already in NonOperational status for reason ARCHITECTURE_INCOMPATIBLE_WITH_CLUSTER. SetNonOperationalVds command is skipped. >2014-10-20 14:19:51,162 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-7-thread-14) [6f79edae] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: d5b2492d-8110-4614-9ce4-4a7a118c7942 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-10-20 14:19:51,165 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-14) [6f79edae] START, SetVdsStatusVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, status=PreparingForMaintenance, nonOperationalReason=NONE, stopSpmFailureLogged=true), log id: 5b6dbdf2 >2014-10-20 14:19:51,169 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-14) [6f79edae] FINISH, SetVdsStatusVDSCommand, log id: 5b6dbdf2 >2014-10-20 14:19:51,335 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (org.ovirt.thread.pool-7-thread-14) [6f79edae] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: d5b2492d-8110-4614-9ce4-4a7a118c7942 Type: VDS >2014-10-20 14:19:51,341 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-14) [6f79edae] Correlation ID: 6f79edae, Job ID: 94522191-ea40-4dd7-a517-99cf14e9587f, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com was switched to Maintenance mode by admin. >2014-10-20 14:19:53,420 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-88) Updated vds status from Preparing for Maintenance to Maintenance in database, vds = d5b2492d-8110-4614-9ce4-4a7a118c7942 : alma07.qa.lab.tlv.redhat.com >2014-10-20 14:19:53,444 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-7-thread-17) Clearing cache of pool: f8f5eaee-8fd0-4b45-87db-62d61b03a916 for problematic entities of VDS: alma07.qa.lab.tlv.redhat.com. >2014-10-20 14:19:53,444 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-7-thread-17) Removing vds [d5b2492d-8110-4614-9ce4-4a7a118c7942] from the domain in maintenance cache >2014-10-20 14:19:53,444 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-7-thread-17) Removing host(s) [d5b2492d-8110-4614-9ce4-4a7a118c7942] from hosts unseen domain report cache >2014-10-20 14:19:53,447 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-88) START, DisconnectStoragePoolVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, vds_spm_id = 3), log id: 7a85328 >2014-10-20 14:19:53,451 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-88) FINISH, DisconnectStoragePoolVDSCommand, log id: 7a85328 >2014-10-20 14:19:53,453 INFO [org.ovirt.engine.core.bll.storage.DisconnectHostFromStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-88) [44e9efe] Running command: DisconnectHostFromStoragePoolServersCommand internal: true. Entities affected : ID: f8f5eaee-8fd0-4b45-87db-62d61b03a916 Type: StoragePool >2014-10-20 14:19:53,458 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-88) [44e9efe] START, DisconnectStorageServerVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, storageType = NFS, connectionList = [{ id: 5eec92db-0201-4aa5-8598-1fe041c7a057, connection: 10.35.160.108:/RHEV/artyom_storage_rhevm_35, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 19bb35d5 >2014-10-20 14:19:53,605 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-88) [44e9efe] FINISH, DisconnectStorageServerVDSCommand, return: {5eec92db-0201-4aa5-8598-1fe041c7a057=477}, log id: 19bb35d5 >2014-10-20 14:21:29,704 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcherImpl.read0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) [rt.jar:1.7.0_65] > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) [rt.jar:1.7.0_65] > at sun.nio.ch.IOUtil.read(IOUtil.java:197) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.read(SSLEngineNioHelper.java:51) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.read(SSLClient.java:81) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient.processIncoming(StompCommonClient.java:90) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.process(ReactorClient.java:168) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:115) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] > >2014-10-20 14:21:37,991 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (ajp-/127.0.0.1:8702-2) [16e57567] Lock Acquired to object EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS >, sharedLocks= ] >2014-10-20 14:21:38,018 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-7-thread-28) [16e57567] Running command: ActivateVdsCommand internal: false. Entities affected : ID: d5b2492d-8110-4614-9ce4-4a7a118c7942 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-10-20 14:21:38,018 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-7-thread-28) [16e57567] Before acquiring lock in order to prevent monitoring for host alma07.qa.lab.tlv.redhat.com from data-center dc_35 >2014-10-20 14:21:38,018 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-7-thread-28) [16e57567] Lock acquired, from now a monitoring of host will be skipped for host alma07.qa.lab.tlv.redhat.com from data-center dc_35 >2014-10-20 14:21:38,028 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-28) [16e57567] START, SetVdsStatusVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, status=Unassigned, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 275be124 >2014-10-20 14:21:38,032 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-28) [16e57567] FINISH, SetVdsStatusVDSCommand, log id: 275be124 >2014-10-20 14:21:38,037 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-7-thread-28) [16e57567] Activate finished. Lock released. Monitoring can run now for host alma07.qa.lab.tlv.redhat.com from data-center dc_35 >2014-10-20 14:21:38,053 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-28) [16e57567] Correlation ID: 16e57567, Job ID: ab244b63-51d2-42bb-9e3e-d6df43e542ad, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com was activated by admin. >2014-10-20 14:21:38,057 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-7-thread-28) [16e57567] Lock freed to object EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS >, sharedLocks= ] >2014-10-20 14:21:38,716 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-20 14:21:39,140 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-80) START, GetHardwareInfoVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]), log id: 2f85442d >2014-10-20 14:21:39,144 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-80) FINISH, GetHardwareInfoVDSCommand, log id: 2f85442d >2014-10-20 14:21:39,174 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-80) [1f3347fb] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: d5b2492d-8110-4614-9ce4-4a7a118c7942 Type: VDS >2014-10-20 14:21:39,286 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-80) [614f19bb] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: d5b2492d-8110-4614-9ce4-4a7a118c7942 Type: VDS >2014-10-20 14:21:39,288 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-80) [614f19bb] START, SetVdsStatusVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, status=NonOperational, nonOperationalReason=EMULATED_MACHINES_INCOMPATIBLE_WITH_CLUSTER, stopSpmFailureLogged=false), log id: 5dcc90e >2014-10-20 14:21:39,313 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-80) [614f19bb] FINISH, SetVdsStatusVDSCommand, log id: 5dcc90e >2014-10-20 14:21:39,333 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-80) [614f19bb] Correlation ID: 614f19bb, Job ID: 305cc1d5-3199-499c-9290-288e5af82f83, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com does not comply with the cluster cl_35_ppc emulated machines. The Hosts emulated machines are <UNKNOWN> and the cluster is rhel6.5.0} >2014-10-20 14:21:39,346 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-80) [614f19bb] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host alma07.qa.lab.tlv.redhat.com was set to NonOperational. >2014-10-20 14:21:39,436 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-80) [600576d2] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: d5b2492d-8110-4614-9ce4-4a7a118c7942 Type: VDS >2014-10-20 14:21:39,440 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-80) [da2c377] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: d5b2492d-8110-4614-9ce4-4a7a118c7942 Type: VDS >2014-10-20 14:21:39,442 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-80) [da2c377] Host d5b2492d-8110-4614-9ce4-4a7a118c7942 : alma07.qa.lab.tlv.redhat.com is already in NonOperational status for reason EMULATED_MACHINES_INCOMPATIBLE_WITH_CLUSTER. SetNonOperationalVds command is skipped. >2014-10-20 14:25:00,012 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-94) Autorecovering 1 hosts >2014-10-20 14:25:00,013 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-94) Autorecovering hosts id: d5b2492d-8110-4614-9ce4-4a7a118c7942, name : alma07.qa.lab.tlv.redhat.com >2014-10-20 14:25:00,016 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-94) [2aabd4a0] Lock Acquired to object EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS >, sharedLocks= ] >2014-10-20 14:25:00,016 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-94) [2aabd4a0] Running command: ActivateVdsCommand internal: true. Entities affected : ID: d5b2492d-8110-4614-9ce4-4a7a118c7942 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-10-20 14:25:00,016 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-94) [2aabd4a0] Before acquiring lock in order to prevent monitoring for host alma07.qa.lab.tlv.redhat.com from data-center dc_35 >2014-10-20 14:25:00,016 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-94) [2aabd4a0] Lock acquired, from now a monitoring of host will be skipped for host alma07.qa.lab.tlv.redhat.com from data-center dc_35 >2014-10-20 14:25:00,027 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-94) [2aabd4a0] START, SetVdsStatusVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, status=Unassigned, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 68687a6a >2014-10-20 14:25:00,031 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-94) [2aabd4a0] FINISH, SetVdsStatusVDSCommand, log id: 68687a6a >2014-10-20 14:25:00,037 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-94) [2aabd4a0] Activate finished. Lock released. Monitoring can run now for host alma07.qa.lab.tlv.redhat.com from data-center dc_35 >2014-10-20 14:25:00,043 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-94) [2aabd4a0] Correlation ID: 2aabd4a0, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com was autorecovered. >2014-10-20 14:25:00,043 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-94) [2aabd4a0] Lock freed to object EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS >, sharedLocks= ] >2014-10-20 14:25:00,683 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-66) START, GetHardwareInfoVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]), log id: 3f0af8ae >2014-10-20 14:25:00,688 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-66) FINISH, GetHardwareInfoVDSCommand, log id: 3f0af8ae >2014-10-20 14:25:00,712 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-66) [471caa63] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: d5b2492d-8110-4614-9ce4-4a7a118c7942 Type: VDS >2014-10-20 14:25:00,737 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-66) [55517166] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: d5b2492d-8110-4614-9ce4-4a7a118c7942 Type: VDS >2014-10-20 14:25:00,739 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-66) [55517166] START, SetVdsStatusVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, status=NonOperational, nonOperationalReason=EMULATED_MACHINES_INCOMPATIBLE_WITH_CLUSTER, stopSpmFailureLogged=false), log id: 58bce1e5 >2014-10-20 14:25:00,742 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-66) [55517166] FINISH, SetVdsStatusVDSCommand, log id: 58bce1e5 >2014-10-20 14:25:00,754 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-66) [55517166] Correlation ID: 55517166, Job ID: 7330f49b-10fa-4db4-8837-12f4dc85be66, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com does not comply with the cluster cl_35_ppc emulated machines. The Hosts emulated machines are <UNKNOWN> and the cluster is rhel6.5.0} >2014-10-20 14:25:00,763 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-66) [55517166] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host alma07.qa.lab.tlv.redhat.com was set to NonOperational. >2014-10-20 14:25:00,787 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-66) [1ccfa751] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: d5b2492d-8110-4614-9ce4-4a7a118c7942 Type: VDS >2014-10-20 14:25:00,789 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-66) [1ccfa751] Host d5b2492d-8110-4614-9ce4-4a7a118c7942 : alma07.qa.lab.tlv.redhat.com is already in NonOperational status for reason EMULATED_MACHINES_INCOMPATIBLE_WITH_CLUSTER. SetNonOperationalVds command is skipped. >2014-10-20 14:30:00,013 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-81) Autorecovering 1 hosts >2014-10-20 14:30:00,014 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-81) Autorecovering hosts id: d5b2492d-8110-4614-9ce4-4a7a118c7942, name : alma07.qa.lab.tlv.redhat.com >2014-10-20 14:30:00,018 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-81) [20b653b7] Lock Acquired to object EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS >, sharedLocks= ] >2014-10-20 14:30:00,018 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-81) [20b653b7] Running command: ActivateVdsCommand internal: true. Entities affected : ID: d5b2492d-8110-4614-9ce4-4a7a118c7942 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-10-20 14:30:00,018 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-81) [20b653b7] Before acquiring lock in order to prevent monitoring for host alma07.qa.lab.tlv.redhat.com from data-center dc_35 >2014-10-20 14:30:00,018 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-81) [20b653b7] Lock acquired, from now a monitoring of host will be skipped for host alma07.qa.lab.tlv.redhat.com from data-center dc_35 >2014-10-20 14:30:00,028 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-81) [20b653b7] START, SetVdsStatusVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, status=Unassigned, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 36f21032 >2014-10-20 14:30:00,032 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-81) [20b653b7] FINISH, SetVdsStatusVDSCommand, log id: 36f21032 >2014-10-20 14:30:00,038 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-81) [20b653b7] Activate finished. Lock released. Monitoring can run now for host alma07.qa.lab.tlv.redhat.com from data-center dc_35 >2014-10-20 14:30:00,041 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-81) [20b653b7] Lock freed to object EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS >, sharedLocks= ] >2014-10-20 14:30:01,140 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-50) [4314b1db] START, GetHardwareInfoVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]), log id: 33ac559c >2014-10-20 14:30:01,145 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-50) [4314b1db] FINISH, GetHardwareInfoVDSCommand, log id: 33ac559c >2014-10-20 14:30:01,171 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-50) [26b3801c] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: d5b2492d-8110-4614-9ce4-4a7a118c7942 Type: VDS >2014-10-20 14:30:01,193 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-50) [1d14c8dd] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: d5b2492d-8110-4614-9ce4-4a7a118c7942 Type: VDS >2014-10-20 14:30:01,196 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-50) [1d14c8dd] START, SetVdsStatusVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, status=NonOperational, nonOperationalReason=EMULATED_MACHINES_INCOMPATIBLE_WITH_CLUSTER, stopSpmFailureLogged=false), log id: 3b1abaea >2014-10-20 14:30:01,199 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-50) [1d14c8dd] FINISH, SetVdsStatusVDSCommand, log id: 3b1abaea >2014-10-20 14:30:01,210 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-50) [1d14c8dd] Correlation ID: 1d14c8dd, Job ID: a06d65a6-93cc-4a4f-af5e-fc7a93f292de, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com does not comply with the cluster cl_35_ppc emulated machines. The Hosts emulated machines are <UNKNOWN> and the cluster is rhel6.5.0} >2014-10-20 14:30:01,220 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-50) [1d14c8dd] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host alma07.qa.lab.tlv.redhat.com was set to NonOperational. >2014-10-20 14:30:01,237 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-50) [1003596] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: d5b2492d-8110-4614-9ce4-4a7a118c7942 Type: VDS >2014-10-20 14:30:01,239 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-50) [1003596] Host d5b2492d-8110-4614-9ce4-4a7a118c7942 : alma07.qa.lab.tlv.redhat.com is already in NonOperational status for reason EMULATED_MACHINES_INCOMPATIBLE_WITH_CLUSTER. SetNonOperationalVds command is skipped. >2014-10-20 14:35:00,011 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-15) Autorecovering 1 hosts >2014-10-20 14:35:00,011 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-15) Autorecovering hosts id: d5b2492d-8110-4614-9ce4-4a7a118c7942, name : alma07.qa.lab.tlv.redhat.com >2014-10-20 14:35:00,015 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-15) [29afdc83] Lock Acquired to object EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS >, sharedLocks= ] >2014-10-20 14:35:00,015 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-15) [29afdc83] Running command: ActivateVdsCommand internal: true. Entities affected : ID: d5b2492d-8110-4614-9ce4-4a7a118c7942 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-10-20 14:35:00,015 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-15) [29afdc83] Before acquiring lock in order to prevent monitoring for host alma07.qa.lab.tlv.redhat.com from data-center dc_35 >2014-10-20 14:35:00,015 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-15) [29afdc83] Lock acquired, from now a monitoring of host will be skipped for host alma07.qa.lab.tlv.redhat.com from data-center dc_35 >2014-10-20 14:35:00,026 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-15) [29afdc83] START, SetVdsStatusVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, status=Unassigned, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 29b2a2b5 >2014-10-20 14:35:00,029 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-15) [29afdc83] FINISH, SetVdsStatusVDSCommand, log id: 29b2a2b5 >2014-10-20 14:35:00,036 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-15) [29afdc83] Activate finished. Lock released. Monitoring can run now for host alma07.qa.lab.tlv.redhat.com from data-center dc_35 >2014-10-20 14:35:00,038 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-15) [29afdc83] Lock freed to object EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS >, sharedLocks= ] >2014-10-20 14:35:01,584 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-45) START, GetHardwareInfoVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]), log id: cce5300 >2014-10-20 14:35:01,588 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-45) FINISH, GetHardwareInfoVDSCommand, log id: cce5300 >2014-10-20 14:35:01,619 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-45) [47d69e2d] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: d5b2492d-8110-4614-9ce4-4a7a118c7942 Type: VDS >2014-10-20 14:35:01,643 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-45) [1da34896] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: d5b2492d-8110-4614-9ce4-4a7a118c7942 Type: VDS >2014-10-20 14:35:01,645 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-45) [1da34896] START, SetVdsStatusVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, status=NonOperational, nonOperationalReason=EMULATED_MACHINES_INCOMPATIBLE_WITH_CLUSTER, stopSpmFailureLogged=false), log id: 1e431bdb >2014-10-20 14:35:01,648 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-45) [1da34896] FINISH, SetVdsStatusVDSCommand, log id: 1e431bdb >2014-10-20 14:35:01,660 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-45) [1da34896] Correlation ID: 1da34896, Job ID: 1fbe5632-4def-4810-8fc3-7dce080a6755, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com does not comply with the cluster cl_35_ppc emulated machines. The Hosts emulated machines are <UNKNOWN> and the cluster is rhel6.5.0} >2014-10-20 14:35:01,669 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-45) [1da34896] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host alma07.qa.lab.tlv.redhat.com was set to NonOperational. >2014-10-20 14:35:01,689 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-45) [737ddb76] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: d5b2492d-8110-4614-9ce4-4a7a118c7942 Type: VDS >2014-10-20 14:35:01,691 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-45) [737ddb76] Host d5b2492d-8110-4614-9ce4-4a7a118c7942 : alma07.qa.lab.tlv.redhat.com is already in NonOperational status for reason EMULATED_MACHINES_INCOMPATIBLE_WITH_CLUSTER. SetNonOperationalVds command is skipped. >2014-10-20 14:35:59,878 INFO [org.ovirt.engine.core.bll.AddVdsGroupCommand] (ajp-/127.0.0.1:8702-2) [2463ff1a] Running command: AddVdsGroupCommand internal: false. Entities affected : ID: f8f5eaee-8fd0-4b45-87db-62d61b03a916 Type: StoragePoolAction group CREATE_CLUSTER with role type ADMIN >2014-10-20 14:35:59,889 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) [2463ff1a] Correlation ID: 2463ff1a, Call Stack: null, Custom Event ID: -1, Message: Cluster test was added by admin >2014-10-20 14:38:24,153 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-3) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 14:40:00,012 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-76) Autorecovering 1 hosts >2014-10-20 14:40:00,012 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-76) Autorecovering hosts id: d5b2492d-8110-4614-9ce4-4a7a118c7942, name : alma07.qa.lab.tlv.redhat.com >2014-10-20 14:40:00,015 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-76) [186c0675] Lock Acquired to object EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS >, sharedLocks= ] >2014-10-20 14:40:00,015 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-76) [186c0675] Running command: ActivateVdsCommand internal: true. Entities affected : ID: d5b2492d-8110-4614-9ce4-4a7a118c7942 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-10-20 14:40:00,016 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-76) [186c0675] Before acquiring lock in order to prevent monitoring for host alma07.qa.lab.tlv.redhat.com from data-center dc_35 >2014-10-20 14:40:00,017 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-76) [186c0675] Lock acquired, from now a monitoring of host will be skipped for host alma07.qa.lab.tlv.redhat.com from data-center dc_35 >2014-10-20 14:40:00,021 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-76) [186c0675] START, SetVdsStatusVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, status=Unassigned, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 3472a624 >2014-10-20 14:40:00,025 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-76) [186c0675] FINISH, SetVdsStatusVDSCommand, log id: 3472a624 >2014-10-20 14:40:00,030 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-76) [186c0675] Activate finished. Lock released. Monitoring can run now for host alma07.qa.lab.tlv.redhat.com from data-center dc_35 >2014-10-20 14:40:00,031 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-76) [186c0675] Lock freed to object EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS >, sharedLocks= ] >2014-10-20 14:40:06,129 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcherImpl.write0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47) [rt.jar:1.7.0_65] > at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93) [rt.jar:1.7.0_65] > at sun.nio.ch.IOUtil.write(IOUtil.java:65) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:487) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:95) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.write(SSLClient.java:90) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.processOutgoing(ReactorClient.java:199) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.process(ReactorClient.java:170) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:115) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] > >2014-10-20 14:40:16,029 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (ajp-/127.0.0.1:8702-2) [5bca0fde] Lock Acquired to object EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS >, sharedLocks= ] >2014-10-20 14:40:16,062 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Running command: ActivateVdsCommand internal: false. Entities affected : ID: d5b2492d-8110-4614-9ce4-4a7a118c7942 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-10-20 14:40:16,062 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Before acquiring lock in order to prevent monitoring for host alma07.qa.lab.tlv.redhat.com from data-center dc_35 >2014-10-20 14:40:16,062 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:18,264 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:18,342 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:18,513 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:18,543 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:21,281 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:21,368 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:21,546 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:21,580 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:24,299 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:24,395 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:24,582 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:24,616 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:27,364 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:27,482 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:27,614 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:27,645 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:30,388 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:30,511 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:30,646 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:30,677 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:33,404 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:33,569 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:33,680 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:33,735 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:36,419 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:36,594 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:36,713 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:36,771 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:39,435 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:39,620 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:39,746 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:39,804 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:42,499 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:42,706 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:42,778 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:42,839 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:45,517 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:45,735 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:45,813 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:45,872 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:48,532 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:48,760 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:48,846 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:48,905 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:51,549 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:51,784 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:51,877 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:51,938 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:54,566 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:54,811 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:54,914 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:54,969 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:57,630 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:57,893 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:57,948 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:40:58,001 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:00,655 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:00,921 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:00,980 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:01,033 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:03,685 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:03,946 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:04,013 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:04,065 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:06,700 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:06,968 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:07,045 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:07,097 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:09,716 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:09,994 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:10,077 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:10,129 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:12,780 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:13,073 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:13,109 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:13,162 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:15,799 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:16,101 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:16,140 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:16,194 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:18,818 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:19,127 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:19,174 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:19,226 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:21,834 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:22,150 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:22,207 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:22,258 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:24,851 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:25,196 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:25,243 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:25,294 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:27,920 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:28,278 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:28,279 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:28,326 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:30,952 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:31,310 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:31,311 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:31,363 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:33,971 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:34,332 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:34,342 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:34,395 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:36,987 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:37,356 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:37,375 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:37,429 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:40,003 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:40,381 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:40,408 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:40,461 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:43,071 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:43,441 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:43,476 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:43,494 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:46,089 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:46,473 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:46,503 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:46,527 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:49,106 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:49,505 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:49,529 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:49,559 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:52,123 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:52,542 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:52,599 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:52,600 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:55,141 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:55,574 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:55,623 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:55,632 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:58,212 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:58,607 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:58,666 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:41:58,693 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:01,230 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:01,639 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:01,697 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:01,722 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:04,246 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:04,671 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:04,730 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:04,747 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:07,261 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:07,704 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:07,763 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:07,772 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:10,279 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:10,736 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:10,795 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:10,796 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:13,347 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:13,768 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:13,827 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:13,867 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:16,364 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:16,802 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:16,859 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:16,895 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:19,381 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:19,837 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:19,892 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:19,924 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:22,399 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:22,871 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:22,924 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:22,951 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:25,415 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:25,904 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:25,960 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:26,020 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:28,482 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:28,937 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:28,996 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:29,102 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:31,498 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:31,969 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:32,031 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:32,126 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:34,515 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:35,001 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:35,064 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:35,153 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:37,531 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:38,036 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:38,097 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:38,175 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:38,205 INFO [org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand] (ajp-/127.0.0.1:8702-5) [7e02dd35] Lock Acquired to object EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_FENCE >, sharedLocks= ] >2014-10-20 14:42:38,205 WARN [org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand] (ajp-/127.0.0.1:8702-5) [7e02dd35] CanDoAction of action FenceVdsManualy failed. Reasons:VAR__ACTION__MANUAL_FENCE,VAR__TYPE__HOST,ACTION_TYPE_FAILED_VDS_NOT_MATCH_VALID_STATUS >2014-10-20 14:42:38,206 INFO [org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand] (ajp-/127.0.0.1:8702-5) [7e02dd35] Lock freed to object EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_FENCE >, sharedLocks= ] >2014-10-20 14:42:38,207 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:40,550 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:41,068 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:41,130 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:41,200 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:43,626 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:44,099 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:44,162 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:44,276 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:46,643 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:47,134 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:47,193 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:47,303 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:49,661 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:50,170 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:50,227 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:50,330 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:52,678 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:53,198 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:53,259 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:53,355 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:55,696 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:56,230 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:56,293 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:56,380 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:58,763 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:59,264 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:59,325 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:42:59,457 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:43:01,780 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_INIT >, sharedLocks= ] >2014-10-20 14:43:02,195 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-10) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@77780820 >2014-10-20 14:43:02,195 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-10) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-20 14:43:02,195 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-10) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-20 14:43:02,195 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-10) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 60 seconds and after that an attempt to fence the host will be issued. >2014-10-20 14:43:02,202 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-10) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 60 seconds and after that an attempt to fence the host will be issued. >2014-10-20 14:43:02,206 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-10) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-20 14:43:02,210 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-10) Failed to refresh VDS , vds = d5b2492d-8110-4614-9ce4-4a7a118c7942 : alma07.qa.lab.tlv.redhat.com, VDS Network Error, continuing. >VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-20 14:43:02,210 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Lock acquired, from now a monitoring of host will be skipped for host alma07.qa.lab.tlv.redhat.com from data-center dc_35 >2014-10-20 14:43:02,215 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] START, SetVdsStatusVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, status=Unassigned, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 7d35c1a4 >2014-10-20 14:43:02,218 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] FINISH, SetVdsStatusVDSCommand, log id: 7d35c1a4 >2014-10-20 14:43:02,224 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Activate finished. Lock released. Monitoring can run now for host alma07.qa.lab.tlv.redhat.com from data-center dc_35 >2014-10-20 14:43:02,228 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Correlation ID: 5bca0fde, Job ID: 232f63a4-0b20-4c9f-b1de-209c3ee6d803, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com was activated by admin. >2014-10-20 14:43:02,231 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-7-thread-50) [5bca0fde] Lock freed to object EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS >, sharedLocks= ] >2014-10-20 14:43:05,351 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-20 14:43:05,731 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-34) START, GetHardwareInfoVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]), log id: 7349b6ff >2014-10-20 14:43:05,737 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-34) FINISH, GetHardwareInfoVDSCommand, log id: 7349b6ff >2014-10-20 14:43:05,770 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-34) [60e218fb] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: d5b2492d-8110-4614-9ce4-4a7a118c7942 Type: VDS >2014-10-20 14:43:05,780 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-34) [475e0b27] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: f8f5eaee-8fd0-4b45-87db-62d61b03a916 Type: StoragePool >2014-10-20 14:43:05,789 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-34) [12af9d3c] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: f8f5eaee-8fd0-4b45-87db-62d61b03a916 Type: StoragePool >2014-10-20 14:43:05,799 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-34) [12af9d3c] START, ConnectStorageServerVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, storageType = NFS, connectionList = [{ id: 5eec92db-0201-4aa5-8598-1fe041c7a057, connection: 10.35.160.108:/RHEV/artyom_storage_rhevm_35, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 182c9d32 >2014-10-20 14:43:06,106 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-34) [12af9d3c] FINISH, ConnectStorageServerVDSCommand, return: {5eec92db-0201-4aa5-8598-1fe041c7a057=0}, log id: 182c9d32 >2014-10-20 14:43:06,106 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-34) [12af9d3c] Host alma07.qa.lab.tlv.redhat.com storage connection was succeeded >2014-10-20 14:43:06,114 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-7-thread-17) START, ConnectStoragePoolVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vdsId = d5b2492d-8110-4614-9ce4-4a7a118c7942, storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, masterVersion = 1), log id: 4dfd730a >2014-10-20 14:43:06,404 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-7-thread-17) FINISH, ConnectStoragePoolVDSCommand, log id: 4dfd730a >2014-10-20 14:43:06,418 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-34) [12af9d3c] START, SetMOMPolicyParametersVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942), log id: 467be049 >2014-10-20 14:43:06,479 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-34) [12af9d3c] FINISH, SetMOMPolicyParametersVDSCommand, log id: 467be049 >2014-10-20 14:43:06,485 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-34) [12af9d3c] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host alma07.qa.lab.tlv.redhat.com was set to Up. >2014-10-20 14:43:06,490 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-34) [12af9d3c] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host alma07.qa.lab.tlv.redhat.com. >2014-10-20 14:43:06,511 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-34) [c04f6c7] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: d5b2492d-8110-4614-9ce4-4a7a118c7942 Type: VDS >2014-10-20 14:44:00,624 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcherImpl.write0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47) [rt.jar:1.7.0_65] > at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93) [rt.jar:1.7.0_65] > at sun.nio.ch.IOUtil.write(IOUtil.java:65) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:487) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:95) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.write(SSLClient.java:90) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.processOutgoing(ReactorClient.java:199) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.process(ReactorClient.java:170) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:115) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] > >2014-10-20 14:44:00,625 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcherImpl.read0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) [rt.jar:1.7.0_65] > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) [rt.jar:1.7.0_65] > at sun.nio.ch.IOUtil.read(IOUtil.java:197) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.read(SSLEngineNioHelper.java:51) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.read(SSLClient.java:81) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient.processIncoming(StompCommonClient.java:90) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.process(ReactorClient.java:168) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:115) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] > >2014-10-20 14:44:41,286 INFO [org.ovirt.engine.core.bll.AddVmFromScratchCommand] (ajp-/127.0.0.1:8702-2) [46407f06] Lock Acquired to object EngineLock [exclusiveLocks= key: ppc_vm value: VM_NAME >, sharedLocks= ] >2014-10-20 14:44:41,339 INFO [org.ovirt.engine.core.bll.AddVmFromScratchCommand] (ajp-/127.0.0.1:8702-2) [46407f06] Running command: AddVmFromScratchCommand internal: false. Entities affected : ID: 63702093-c04f-417c-8045-aa56334b54ee Type: VdsGroupsAction group CREATE_VM with role type USER >2014-10-20 14:44:41,404 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) [46407f06] Correlation ID: 46407f06, Job ID: 135e1b62-4c4a-4803-a420-a4eeab2f1e7c, Call Stack: null, Custom Event ID: -1, Message: VM ppc_vm was created by admin. >2014-10-20 14:44:41,408 INFO [org.ovirt.engine.core.bll.AddVmFromScratchCommand] (ajp-/127.0.0.1:8702-2) [46407f06] Lock freed to object EngineLock [exclusiveLocks= key: ppc_vm value: VM_NAME >, sharedLocks= ] >2014-10-20 14:44:42,022 INFO [org.ovirt.engine.core.bll.network.vm.AddVmInterfaceCommand] (ajp-/127.0.0.1:8702-1) [4407bbe0] Running command: AddVmInterfaceCommand internal: false. Entities affected : ID: 17b1e1e6-7dbe-4a3d-b556-685ca579d0e2 Type: VMAction group CONFIGURE_VM_NETWORK with role type USER, ID: c078ca43-dc90-4ffe-a17e-68721438ee95 Type: VnicProfileAction group CONFIGURE_VM_NETWORK with role type USER >2014-10-20 14:44:42,059 INFO [org.ovirt.engine.core.bll.network.vm.ActivateDeactivateVmNicCommand] (ajp-/127.0.0.1:8702-1) [611e710b] Running command: ActivateDeactivateVmNicCommand internal: true. Entities affected : ID: 17b1e1e6-7dbe-4a3d-b556-685ca579d0e2 Type: VM >2014-10-20 14:44:42,095 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [611e710b] Correlation ID: 611e710b, Call Stack: null, Custom Event ID: -1, Message: Network Interface nic1 (VirtIO) was plugged to VM ppc_vm. (User: admin) >2014-10-20 14:44:42,102 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [611e710b] Correlation ID: 4407bbe0, Call Stack: null, Custom Event ID: -1, Message: Interface nic1 (VirtIO) was added to VM ppc_vm. (User: admin) >2014-10-20 14:44:42,139 INFO [org.ovirt.engine.core.bll.network.vm.ReorderVmNicsCommand] (ajp-/127.0.0.1:8702-4) [69774e0c] Running command: ReorderVmNicsCommand internal: false. Entities affected : ID: 17b1e1e6-7dbe-4a3d-b556-685ca579d0e2 Type: VMAction group CREATE_VM with role type USER >2014-10-20 14:44:46,162 INFO [org.ovirt.engine.core.bll.AddDiskCommand] (ajp-/127.0.0.1:8702-6) [32aba475] Lock Acquired to object EngineLock [exclusiveLocks= key: 17b1e1e6-7dbe-4a3d-b556-685ca579d0e2 value: VM_DISK_BOOT >, sharedLocks= key: 17b1e1e6-7dbe-4a3d-b556-685ca579d0e2 value: VM >] >2014-10-20 14:44:46,190 INFO [org.ovirt.engine.core.bll.AddDiskCommand] (ajp-/127.0.0.1:8702-6) [32aba475] Running command: AddDiskCommand internal: false. Entities affected : ID: 17b1e1e6-7dbe-4a3d-b556-685ca579d0e2 Type: VMAction group CONFIGURE_VM_STORAGE with role type USER, ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: StorageAction group CREATE_DISK with role type USER >2014-10-20 14:44:46,259 INFO [org.ovirt.engine.core.bll.AddImageFromScratchCommand] (ajp-/127.0.0.1:8702-6) [2cf2b391] Running command: AddImageFromScratchCommand internal: true. Entities affected : ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: Storage >2014-10-20 14:44:46,268 INFO [org.ovirt.engine.core.bll.AddImageFromScratchCommand] (ajp-/127.0.0.1:8702-6) [2cf2b391] Lock freed to object EngineLock [exclusiveLocks= key: 17b1e1e6-7dbe-4a3d-b556-685ca579d0e2 value: VM_DISK_BOOT >, sharedLocks= key: 17b1e1e6-7dbe-4a3d-b556-685ca579d0e2 value: VM >] >2014-10-20 14:44:46,269 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (ajp-/127.0.0.1:8702-6) [2cf2b391] START, CreateImageVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = 40e2fdae-cc93-4242-aa96-1624d7bb788e, imageSizeInBytes = 1073741824, volumeFormat = RAW, newImageId = 79816e30-f79f-4872-a604-c7b077a7846b, newImageDescription = ), log id: 485012d6 >2014-10-20 14:46:57,324 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ListVDSCommand] (DefaultQuartzScheduler_Worker-24) Command ListVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-20 14:46:57,325 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-24) Host alma06.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 80 seconds and after that an attempt to fence the host will be issued. >2014-10-20 14:46:57,332 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-24) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 80 seconds and after that an attempt to fence the host will be issued. >2014-10-20 14:46:57,332 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-24) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.ListVDSCommand.executeVdsBrokerCommand(ListVDSCommand.java:24) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:418) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.fetchRunningVms(VdsUpdateRunTimeInfo.java:1000) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVmStats(VdsUpdateRunTimeInfo.java:949) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:667) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-20 14:46:57,339 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-24) Failed to refresh VDS , vds = 1c39bd88-8672-408f-a593-60d6ef8f647c : alma06.qa.lab.tlv.redhat.com, VDS Network Error, continuing. >VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-20 14:46:57,826 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-61) Command SpmStatusVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-20 14:46:57,829 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-61) [3460e99b] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: f8f5eaee-8fd0-4b45-87db-62d61b03a916 Type: StoragePool >2014-10-20 14:46:57,831 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-61) [3460e99b] Storage Pool f8f5eaee-8fd0-4b45-87db-62d61b03a916 - Updating Storage Domain a1341057-9cf8-4750-a734-656d05a5d2c5 status from Active to Unknown, reason : null >2014-10-20 14:46:57,841 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-61) [3460e99b] Correlation ID: 3460e99b, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center dc_35. Setting Data Center status to Non Responsive (On host alma06.qa.lab.tlv.redhat.com, Error: Network error during communication with the Host.). >2014-10-20 14:46:57,979 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-20 14:46:58,188 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.GetStoragePoolInfoVDSCommand] (DefaultQuartzScheduler_Worker-61) [3460e99b] Failed in GetStoragePoolInfoVDS method >2014-10-20 14:46:58,188 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-61) [3460e99b] IrsBroker::Failed::GetStoragePoolInfoVDS due to: IRSErrorException: IRSGenericException: IRSErrorException: Failed to GetStoragePoolInfoVDS, error = Unknown pool id, pool not connected: (u'f8f5eaee-8fd0-4b45-87db-62d61b03a916',), code = 309 >2014-10-20 14:46:58,192 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-20 14:46:58,274 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-61) [3460e99b] Irs placed on server 1c39bd88-8672-408f-a593-60d6ef8f647c failed. Proceed Failover >2014-10-20 14:46:58,312 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-61) [3460e99b] hostFromVds::selectedVds - cyan-vdsg.qa.lab.tlv.redhat.com, spmStatus Free, storage pool dc_35 >2014-10-20 14:46:58,319 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-61) [3460e99b] starting spm on vds cyan-vdsg.qa.lab.tlv.redhat.com, storage pool dc_35, prevId -1, LVER -1 >2014-10-20 14:46:58,321 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-61) [3460e99b] START, SpmStartVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, prevId=-1, prevLVER=-1, storagePoolFormatType=V3, recoveryMode=Manual, SCSIFencing=false), log id: 5bdafc94 >2014-10-20 14:46:58,339 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-61) [3460e99b] spmStart polling started: taskId = fe9d4114-fdc0-46d9-acc9-fc4fad6d8248 >2014-10-20 14:46:59,349 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-61) [3460e99b] spmStart polling ended: taskId = fe9d4114-fdc0-46d9-acc9-fc4fad6d8248 task status = finished >2014-10-20 14:46:59,372 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-61) [3460e99b] spmStart polling ended, spm status: SPM >2014-10-20 14:46:59,374 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (DefaultQuartzScheduler_Worker-61) [3460e99b] START, HSMClearTaskVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, taskId=fe9d4114-fdc0-46d9-acc9-fc4fad6d8248), log id: 588cd37 >2014-10-20 14:46:59,382 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (DefaultQuartzScheduler_Worker-61) [3460e99b] FINISH, HSMClearTaskVDSCommand, log id: 588cd37 >2014-10-20 14:46:59,382 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-61) [3460e99b] FINISH, SpmStartVDSCommand, return: org.ovirt.engine.core.common.businessentities.SpmStatusResult@627d0ffb, log id: 5bdafc94 >2014-10-20 14:46:59,385 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-61) [3460e99b] Initialize Irs proxy from vds: 10.35.70.4 >2014-10-20 14:46:59,390 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-61) [3460e99b] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Storage Pool Manager runs on Host cyan-vdsg.qa.lab.tlv.redhat.com (Address: 10.35.70.4). >2014-10-20 14:46:59,391 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to /10.35.70.4 >2014-10-20 14:46:59,403 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-44) [3460e99b] START, SPMGetAllTasksInfoVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false), log id: 18b9ca2f >2014-10-20 14:46:59,507 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-44) [3460e99b] -- executeIrsBrokerCommand: Attempting on storage pool f8f5eaee-8fd0-4b45-87db-62d61b03a916 >2014-10-20 14:46:59,508 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-44) [3460e99b] START, HSMGetAllTasksInfoVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6), log id: 8bed664 >2014-10-20 14:46:59,517 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-44) [3460e99b] FINISH, HSMGetAllTasksInfoVDSCommand, return: [], log id: 8bed664 >2014-10-20 14:46:59,517 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-44) [3460e99b] FINISH, SPMGetAllTasksInfoVDSCommand, return: [], log id: 18b9ca2f >2014-10-20 14:46:59,517 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-7-thread-44) [3460e99b] Discovered no tasks on Storage Pool dc_35 >2014-10-20 14:46:59,518 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (ajp-/127.0.0.1:8702-6) [2cf2b391] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-10-20 14:46:59,681 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (ajp-/127.0.0.1:8702-6) [2cf2b391] FINISH, CreateImageVDSCommand, return: 79816e30-f79f-4872-a604-c7b077a7846b, log id: 485012d6 >2014-10-20 14:46:59,686 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (ajp-/127.0.0.1:8702-6) [2cf2b391] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command b0dfed3c-ad79-4de5-a889-faccd3ac7c78 >2014-10-20 14:46:59,686 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (ajp-/127.0.0.1:8702-6) [2cf2b391] CommandMultiAsyncTasks::AttachTask: Attaching task 5de68640-e3b0-4672-a819-f9a1d9fd766c to command b0dfed3c-ad79-4de5-a889-faccd3ac7c78. >2014-10-20 14:46:59,701 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (ajp-/127.0.0.1:8702-6) [2cf2b391] Adding task 5de68640-e3b0-4672-a819-f9a1d9fd766c (Parent Command AddDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-10-20 14:46:59,753 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) [2cf2b391] Correlation ID: 32aba475, Job ID: 7e039d82-1e7e-4f1c-8394-06325b3360ae, Call Stack: null, Custom Event ID: -1, Message: Add-Disk operation of ppc_vm_Disk1 was initiated on VM ppc_vm by admin. >2014-10-20 14:46:59,754 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (ajp-/127.0.0.1:8702-6) [2cf2b391] BaseAsyncTask::startPollingTask: Starting to poll task 5de68640-e3b0-4672-a819-f9a1d9fd766c. >2014-10-20 14:47:00,746 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-20) [721a74f9] START, GetHardwareInfoVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c]), log id: 120cb0b2 >2014-10-20 14:47:00,823 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-20) [721a74f9] FINISH, GetHardwareInfoVDSCommand, log id: 120cb0b2 >2014-10-20 14:47:00,844 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-20) [2046b45b] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 1c39bd88-8672-408f-a593-60d6ef8f647c Type: VDS >2014-10-20 14:47:00,861 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-20) [73387cae] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: f8f5eaee-8fd0-4b45-87db-62d61b03a916 Type: StoragePool >2014-10-20 14:47:00,869 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-20) [5feaad18] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: f8f5eaee-8fd0-4b45-87db-62d61b03a916 Type: StoragePool >2014-10-20 14:47:00,874 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-20) [5feaad18] START, ConnectStorageServerVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, storageType = NFS, connectionList = [{ id: 5eec92db-0201-4aa5-8598-1fe041c7a057, connection: 10.35.160.108:/RHEV/artyom_storage_rhevm_35, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 2b99fb7e >2014-10-20 14:47:00,889 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-20) [5feaad18] FINISH, ConnectStorageServerVDSCommand, return: {5eec92db-0201-4aa5-8598-1fe041c7a057=0}, log id: 2b99fb7e >2014-10-20 14:47:00,889 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-20) [5feaad18] Host alma06.qa.lab.tlv.redhat.com storage connection was succeeded >2014-10-20 14:47:00,914 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-7-thread-30) START, ConnectStoragePoolVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vdsId = 1c39bd88-8672-408f-a593-60d6ef8f647c, storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, masterVersion = 1), log id: 1c999c11 >2014-10-20 14:47:01,257 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-7-thread-30) FINISH, ConnectStoragePoolVDSCommand, log id: 1c999c11 >2014-10-20 14:47:01,279 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-20) [5feaad18] START, SetMOMPolicyParametersVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c), log id: 1992bdfa >2014-10-20 14:47:01,360 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-20) [5feaad18] FINISH, SetMOMPolicyParametersVDSCommand, log id: 1992bdfa >2014-10-20 14:47:01,366 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [5feaad18] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host alma06.qa.lab.tlv.redhat.com was set to Up. >2014-10-20 14:47:01,374 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [5feaad18] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host alma06.qa.lab.tlv.redhat.com. >2014-10-20 14:47:01,394 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-20) [335a42c4] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 1c39bd88-8672-408f-a593-60d6ef8f647c Type: VDS >2014-10-20 14:47:07,024 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-98) Polling and updating Async Tasks: 1 tasks, 1 tasks to poll now >2014-10-20 14:47:07,036 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-98) SPMAsyncTask::PollTask: Polling task 5de68640-e3b0-4672-a819-f9a1d9fd766c (Parent Command AddDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-10-20 14:47:07,041 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-98) BaseAsyncTask::onTaskEndSuccess: Task 5de68640-e3b0-4672-a819-f9a1d9fd766c (Parent Command AddDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-10-20 14:47:07,042 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-98) CommandAsyncTask::endActionIfNecessary: All tasks of command b0dfed3c-ad79-4de5-a889-faccd3ac7c78 has ended -> executing endAction >2014-10-20 14:47:07,043 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-98) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: b0dfed3c-ad79-4de5-a889-faccd3ac7c78): calling endAction . >2014-10-20 14:47:07,043 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-20) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction AddDisk, executionIndex: 0 >2014-10-20 14:47:07,062 INFO [org.ovirt.engine.core.bll.AddDiskCommand] (org.ovirt.thread.pool-7-thread-20) [32aba475] Ending command successfully: org.ovirt.engine.core.bll.AddDiskCommand >2014-10-20 14:47:07,088 INFO [org.ovirt.engine.core.bll.AddImageFromScratchCommand] (org.ovirt.thread.pool-7-thread-20) [2cf2b391] Ending command successfully: org.ovirt.engine.core.bll.AddImageFromScratchCommand >2014-10-20 14:47:07,100 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-20) [2cf2b391] START, GetImageInfoVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = 40e2fdae-cc93-4242-aa96-1624d7bb788e, imageId = 79816e30-f79f-4872-a604-c7b077a7846b), log id: 58ff321d >2014-10-20 14:47:07,147 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-20) [2cf2b391] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@fe130e51, log id: 58ff321d >2014-10-20 14:47:07,162 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-20) Correlation ID: 32aba475, Job ID: 7e039d82-1e7e-4f1c-8394-06325b3360ae, Call Stack: null, Custom Event ID: -1, Message: The disk ppc_vm_Disk1 was successfully added to VM ppc_vm. >2014-10-20 14:47:07,162 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-20) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddDisk completed, handling the result. >2014-10-20 14:47:07,163 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-20) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddDisk succeeded, clearing tasks. >2014-10-20 14:47:07,165 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-20) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 5de68640-e3b0-4672-a819-f9a1d9fd766c >2014-10-20 14:47:07,166 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-20) START, SPMClearTaskVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, taskId = 5de68640-e3b0-4672-a819-f9a1d9fd766c), log id: 1751b6de >2014-10-20 14:47:07,169 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-20) START, HSMClearTaskVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, taskId=5de68640-e3b0-4672-a819-f9a1d9fd766c), log id: 3800b134 >2014-10-20 14:47:07,205 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-20) FINISH, HSMClearTaskVDSCommand, log id: 3800b134 >2014-10-20 14:47:07,205 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-20) FINISH, SPMClearTaskVDSCommand, log id: 1751b6de >2014-10-20 14:47:07,209 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-20) BaseAsyncTask::removeTaskFromDB: Removed task 5de68640-e3b0-4672-a819-f9a1d9fd766c from DataBase >2014-10-20 14:47:07,209 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-20) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity b0dfed3c-ad79-4de5-a889-faccd3ac7c78 >2014-10-20 14:47:53,308 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Loaded file "/usr/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf". >2014-10-20 14:47:53,309 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) The file "/etc/ovirt-engine/engine.conf" doesn't exist or isn't readable. Will return an empty set of properties. >2014-10-20 14:47:53,309 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Loaded file "/etc/ovirt-engine/engine.conf.d/10-rhevm-doc.conf". >2014-10-20 14:47:53,310 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-database.conf". >2014-10-20 14:47:53,310 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf". >2014-10-20 14:47:53,311 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf". >2014-10-20 14:47:53,311 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf". >2014-10-20 14:47:53,312 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Loaded file "/etc/ovirt-engine/engine.conf.d/20-branding-rhev.conf". >2014-10-20 14:47:53,312 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Loaded file "/etc/ovirt-engine/engine.conf.d/50-redhat-support-plugin.conf". >2014-10-20 14:47:53,313 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_AJP_ENABLED" is "true". >2014-10-20 14:47:53,313 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_AJP_PORT" is "8702". >2014-10-20 14:47:53,313 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_APPS" is "engine.ear restapi.war legacy_restapi.war "/usr/share/ovirt-engine/branding/rhev-2.brand/applications/rhev.ear" redhat-support-plugin-rhev.ear". >2014-10-20 14:47:53,314 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_CACHE" is "/var/cache/ovirt-engine". >2014-10-20 14:47:53,314 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_CHECK_INTERVAL" is "1000". >2014-10-20 14:47:53,314 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_CONNECTION_TIMEOUT" is "300000". >2014-10-20 14:47:53,314 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_DATABASE" is "engine". >2014-10-20 14:47:53,315 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_DRIVER" is "org.postgresql.Driver". >2014-10-20 14:47:53,315 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_HOST" is "localhost". >2014-10-20 14:47:53,315 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_MAX_CONNECTIONS" is "100". >2014-10-20 14:47:53,315 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_MIN_CONNECTIONS" is "1". >2014-10-20 14:47:53,315 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_PASSWORD" is "***". >2014-10-20 14:47:53,316 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_PORT" is "5432". >2014-10-20 14:47:53,316 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_SECURED" is "False". >2014-10-20 14:47:53,316 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_SECURED_VALIDATION" is "False". >2014-10-20 14:47:53,316 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_URL" is "jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory". >2014-10-20 14:47:53,317 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_USER" is "engine". >2014-10-20 14:47:53,317 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DEBUG_ADDRESS" is "". >2014-10-20 14:47:53,317 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DOC" is "/usr/share/doc/ovirt-engine". >2014-10-20 14:47:53,317 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_ETC" is "/etc/ovirt-engine". >2014-10-20 14:47:53,357 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_EXTENSION_PATH" is "/usr/share/ovirt-engine/extensions.d:/etc/ovirt-engine/extensions.d". >2014-10-20 14:47:53,357 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_FQDN" is "dhcp163-98.scl.lab.tlv.redhat.com". >2014-10-20 14:47:53,357 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_GROUP" is "ovirt". >2014-10-20 14:47:53,358 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_HEAP_MAX" is "1g". >2014-10-20 14:47:53,358 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_HEAP_MIN" is "1g". >2014-10-20 14:47:53,358 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_HTTPS_ENABLED" is "false". >2014-10-20 14:47:53,359 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_HTTPS_PORT" is "None". >2014-10-20 14:47:53,359 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_HTTPS_PROTOCOLS" is "SSLv3,TLSv1,TLSv1.1,TLSv1.2". >2014-10-20 14:47:53,359 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_HTTP_ENABLED" is "false". >2014-10-20 14:47:53,359 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_HTTP_PORT" is "None". >2014-10-20 14:47:53,360 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_JAVA_MODULEPATH" is "/usr/share/ovirt-engine/modules". >2014-10-20 14:47:53,360 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_JVM_ARGS" is " -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/var/log/ovirt-engine/dump"". >2014-10-20 14:47:53,360 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_LOG" is "/var/log/ovirt-engine". >2014-10-20 14:47:53,360 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_LOG_TO_CONSOLE" is "false". >2014-10-20 14:47:53,360 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_MANUAL" is "/usr/share/doc/rhevm-doc/manual". >2014-10-20 14:47:53,361 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PERM_MAX" is "256m". >2014-10-20 14:47:53,361 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PERM_MIN" is "256m". >2014-10-20 14:47:53,361 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PKI" is "/etc/pki/ovirt-engine". >2014-10-20 14:47:53,361 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PKI_CA" is "/etc/pki/ovirt-engine/ca.pem". >2014-10-20 14:47:53,362 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PKI_ENGINE_CERT" is "/etc/pki/ovirt-engine/certs/engine.cer". >2014-10-20 14:47:53,362 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PKI_ENGINE_STORE" is "/etc/pki/ovirt-engine/keys/engine.p12". >2014-10-20 14:47:53,362 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PKI_ENGINE_STORE_ALIAS" is "1". >2014-10-20 14:47:53,362 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PKI_ENGINE_STORE_PASSWORD" is "***". >2014-10-20 14:47:53,363 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PKI_TRUST_STORE" is "/etc/pki/ovirt-engine/.truststore". >2014-10-20 14:47:53,363 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PKI_TRUST_STORE_PASSWORD" is "***". >2014-10-20 14:47:53,363 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PROPERTIES" is " jsse.enableSNIExtension=false "java.security.krb5.conf=/etc/ovirt-engine/krb5.conf"". >2014-10-20 14:47:53,363 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PROXY_ENABLED" is "true". >2014-10-20 14:47:53,364 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PROXY_HTTPS_PORT" is "443". >2014-10-20 14:47:53,364 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PROXY_HTTP_PORT" is "80". >2014-10-20 14:47:53,364 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_REPORTS_BASE_URL" is "". >2014-10-20 14:47:53,364 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_REPORTS_DASHBOARD_URL" is "/flow.html?viewAsDashboardFrame=true". >2014-10-20 14:47:53,364 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_REPORTS_NOT_INSTALLED_URL" is "/ovirt-engine/ReportsNotInstalled.html". >2014-10-20 14:47:53,365 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_REPORTS_PROXY_URL" is "/ovirt/reports-interface". >2014-10-20 14:47:53,365 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_REPORTS_READ_TIMEOUT" is "". >2014-10-20 14:47:53,365 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_REPORTS_RIGHTCLICK_URL" is "". >2014-10-20 14:47:53,365 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_REPORTS_VERIFY_CHAIN" is "true". >2014-10-20 14:47:53,366 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_REPORTS_VERIFY_HOST" is "true". >2014-10-20 14:47:53,366 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_STOP_INTERVAL" is "1". >2014-10-20 14:47:53,366 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_STOP_TIME" is "10". >2014-10-20 14:47:53,366 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_TMP" is "/var/tmp/ovirt-engine". >2014-10-20 14:47:53,366 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_UP_MARK" is "/var/lib/ovirt-engine/engine.up". >2014-10-20 14:47:53,367 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_URI" is "/ovirt-engine". >2014-10-20 14:47:53,367 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_USER" is "ovirt". >2014-10-20 14:47:53,367 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_USR" is "/usr/share/ovirt-engine". >2014-10-20 14:47:53,367 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_VAR" is "/var/lib/ovirt-engine". >2014-10-20 14:47:53,390 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_VERBOSE_GC" is "false". >2014-10-20 14:47:53,390 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "JBOSS_HOME" is "/usr/share/jbossas". >2014-10-20 14:47:53,390 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "JBOSS_RUNTIME" is "/var/lib/ovirt-engine/jboss_runtime". >2014-10-20 14:47:53,391 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "SENSITIVE_KEYS" is ",ENGINE_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD". >2014-10-20 14:48:05,421 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 42) Start initializing Backend >2014-10-20 14:48:06,599 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (ServerService Thread Pool -- 42) Loaded file /etc/ovirt-engine/osinfo.conf.d/00-defaults.properties >2014-10-20 14:48:07,410 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 42) Running ovirt-engine 3.5.0-0.14.beta.el6ev >2014-10-20 14:48:07,410 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ServerService Thread Pool -- 42) Start initializing dictionaries >2014-10-20 14:48:07,414 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ServerService Thread Pool -- 42) Finished initializing dictionaries >2014-10-20 14:48:07,414 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 42) Start initializing AuditLogCleanupManager >2014-10-20 14:48:07,415 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 42) Setting audit cleanup manager to run at: 35 35 3 * * ? >2014-10-20 14:48:07,498 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 42) Finished initializing AuditLogCleanupManager >2014-10-20 14:48:07,498 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 42) Start initializing CommandEntityCleanupManager >2014-10-20 14:48:07,499 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 42) Setting command entity cleanup manager to run at: 35 35 3 * * ? >2014-10-20 14:48:07,501 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 42) Finished initializing CommandEntityCleanupManager >2014-10-20 14:48:07,503 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 42) Start initializing TagsDirector >2014-10-20 14:48:07,558 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 42) Tag root added to tree >2014-10-20 14:48:07,655 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 42) Finished initializing TagsDirector >2014-10-20 14:48:07,656 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (ServerService Thread Pool -- 42) Start initializing IsoDomainListSyncronizer >2014-10-20 14:48:07,665 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (ServerService Thread Pool -- 42) Finished initializing IsoDomainListSyncronizer >2014-10-20 14:48:07,800 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 42) Completed initializing handlers >2014-10-20 14:48:07,892 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 42) Start initializing ErrorTranslatorImpl >2014-10-20 14:48:07,917 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 42) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table. >2014-10-20 14:48:07,918 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 42) Finished initializing ErrorTranslatorImpl >2014-10-20 14:48:07,918 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 42) Start initializing ErrorTranslatorImpl >2014-10-20 14:48:07,924 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 42) Finished initializing ErrorTranslatorImpl >2014-10-20 14:48:07,924 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 42) Mark incomplete jobs as UNKNOWN >2014-10-20 14:48:07,996 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ServerService Thread Pool -- 42) Start initializing JobRepositoryCleanupManager >2014-10-20 14:48:07,997 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ServerService Thread Pool -- 42) Finished initializing JobRepositoryCleanupManager >2014-10-20 14:48:07,997 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ServerService Thread Pool -- 42) Start initializing AutoRecoveryManager >2014-10-20 14:48:07,999 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ServerService Thread Pool -- 42) Finished initializing AutoRecoveryManager >2014-10-20 14:48:08,001 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ServerService Thread Pool -- 42) Start initializing ExecutionMessageDirector >2014-10-20 14:48:08,004 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ServerService Thread Pool -- 42) Finished initializing ExecutionMessageDirector >2014-10-20 14:48:08,104 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 42) Loading extension 'builtin-authn-internal' >2014-10-20 14:48:08,106 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 42) Extension 'builtin-authn-internal' loaded >2014-10-20 14:48:08,112 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 42) Loading extension 'internal' >2014-10-20 14:48:08,114 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 42) Extension 'internal' loaded >2014-10-20 14:48:08,121 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 42) Loading extension 'builtin-authn-qa.lab.tlv.redhat.com' >2014-10-20 14:48:08,144 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 42) Extension 'builtin-authn-qa.lab.tlv.redhat.com' loaded >2014-10-20 14:48:08,146 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 42) Loading extension 'qa.lab.tlv.redhat.com' >2014-10-20 14:48:08,151 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 42) Extension 'qa.lab.tlv.redhat.com' loaded >2014-10-20 14:48:08,152 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 42) Initializing extension 'qa.lab.tlv.redhat.com' >2014-10-20 14:48:08,152 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 42) Extension 'qa.lab.tlv.redhat.com' initialized >2014-10-20 14:48:08,153 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 42) Initializing extension 'builtin-authn-internal' >2014-10-20 14:48:08,153 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 42) Extension 'builtin-authn-internal' initialized >2014-10-20 14:48:08,154 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 42) Initializing extension 'builtin-authn-qa.lab.tlv.redhat.com' >2014-10-20 14:48:08,154 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 42) Extension 'builtin-authn-qa.lab.tlv.redhat.com' initialized >2014-10-20 14:48:08,155 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 42) Initializing extension 'internal' >2014-10-20 14:48:08,155 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 42) Extension 'internal' initialized >2014-10-20 14:48:08,155 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 42) Start of enabled extensions list >2014-10-20 14:48:08,156 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 42) Instance name: 'qa.lab.tlv.redhat.com', Extension name: 'Kerberos/Ldap Authz (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-10-20 14:48:08,156 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 42) Instance name: 'builtin-authn-internal', Extension name: 'Internal Authn (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-10-20 14:48:08,156 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 42) Instance name: 'builtin-authn-qa.lab.tlv.redhat.com', Extension name: 'Kerberos/Ldap Authn (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-10-20 14:48:08,156 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 42) Instance name: 'internal', Extension name: 'Internal Authz (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-10-20 14:48:08,156 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 42) End of enabled extensions list >2014-10-20 14:48:08,497 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (ServerService Thread Pool -- 42) Start initializing DbUserCacheManager >2014-10-20 14:48:08,498 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (ServerService Thread Pool -- 42) Finished initializing DbUserCacheManager >2014-10-20 14:48:08,512 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (ServerService Thread Pool -- 42) Initialization of AsyncTaskManager completed successfully. >2014-10-20 14:48:08,526 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 42) Start initializing ResourceManager >2014-10-20 14:48:08,860 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 42) Entered VdsManager constructor >2014-10-20 14:48:08,914 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 42) Initialize vdsBroker (rose05.qa.lab.tlv.redhat.com,54,321) >2014-10-20 14:48:09,170 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 42) VDS 45869c56-4fbd-466b-977a-81604a392b78 was added to the Resource Manager >2014-10-20 14:48:09,171 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 42) Entered VdsManager constructor >2014-10-20 14:48:09,208 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 42) Initialize vdsBroker (master-vds10.qa.lab.tlv.redhat.com,54,321) >2014-10-20 14:48:09,244 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 42) VDS 51fdbad1-2bd5-47d4-aa62-4054e71fbbbf was added to the Resource Manager >2014-10-20 14:48:09,245 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 42) Entered VdsManager constructor >2014-10-20 14:48:09,288 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 42) Initialize vdsBroker (alma07.qa.lab.tlv.redhat.com,54,321) >2014-10-20 14:48:09,346 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 42) VDS d5b2492d-8110-4614-9ce4-4a7a118c7942 was added to the Resource Manager >2014-10-20 14:48:09,346 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 42) Entered VdsManager constructor >2014-10-20 14:48:09,363 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 42) Initialize vdsBroker (alma06.qa.lab.tlv.redhat.com,54,321) >2014-10-20 14:48:09,370 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 42) VDS 1c39bd88-8672-408f-a593-60d6ef8f647c was added to the Resource Manager >2014-10-20 14:48:09,371 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 42) Entered VdsManager constructor >2014-10-20 14:48:09,388 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 42) Initialize vdsBroker (10.35.70.4,54,321) >2014-10-20 14:48:09,396 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 42) VDS 3a2a45bd-4411-4428-aa45-a514d30d3ea6 was added to the Resource Manager >2014-10-20 14:48:09,434 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 42) Finished initializing ResourceManager >2014-10-20 14:48:09,442 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (ServerService Thread Pool -- 42) Initialization of OvfDataUpdater completed successfully. >2014-10-20 14:48:09,442 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 42) Start scheduling to enable vds load balancer >2014-10-20 14:48:09,443 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 42) Finished scheduling to enable vds load balancer >2014-10-20 14:48:09,443 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 42) Start HA Reservation check >2014-10-20 14:48:09,443 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 42) Finished HA Reservation check >2014-10-20 14:48:09,455 INFO [org.ovirt.engine.core.bll.network.MacPoolManagerRanges] (org.ovirt.thread.pool-7-thread-1) Start initializing MacPoolManagerRanges >2014-10-20 14:48:09,462 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 42) Init VM custom properties utilities >2014-10-20 14:48:09,462 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 42) Init device custom properties utilities >2014-10-20 14:48:09,467 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 42) Initializing Scheduling manager >2014-10-20 14:48:09,505 INFO [org.ovirt.engine.core.bll.network.MacPoolManagerRanges] (org.ovirt.thread.pool-7-thread-1) Finished initializing. Available MACs in pool: 248 >2014-10-20 14:48:09,542 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 42) Starting external scheduler dicovery thread >2014-10-20 14:48:09,542 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 42) Initialized Scheduling manager >2014-10-20 14:48:09,543 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (ServerService Thread Pool -- 42) Initializing DWH Heart Beat >2014-10-20 14:48:09,544 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (ServerService Thread Pool -- 42) DWH Heart Beat initialized >2014-10-20 14:48:09,674 INFO [org.ovirt.engine.core.bll.scheduling.external.ExternalSchedulerDiscoveryThread] (Thread-69) PolicyUnits updated >2014-10-20 14:48:12,362 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-7) Initializing Host: rose05.qa.lab.tlv.redhat.com >2014-10-20 14:48:12,468 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-8) Initializing Host: master-vds10.qa.lab.tlv.redhat.com >2014-10-20 14:48:12,677 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-20 14:48:12,814 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-20 14:48:12,815 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to /10.35.70.4 >2014-10-20 14:48:13,678 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) AuditLogType: UNASSIGNED not exist in string table >2014-10-20 14:48:13,678 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) AuditLogType: VDS_HIGH_NETWORK_USE not exist in string table >2014-10-20 14:48:13,679 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) AuditLogType: USER_FAILED_REMOVE_VM not exist in string table >2014-10-20 14:48:13,679 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) AuditLogType: USER_RUN_UNLOCK_ENTITY_SCRIPT not exist in string table >2014-10-20 14:48:13,680 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) AuditLogType: VDS_NETWORK_MTU_DIFFER_FROM_LOGICAL_NETWORK not exist in string table >2014-10-20 14:48:13,681 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) AuditLogType: STORAGE_ACTIVATE_ASYNC not exist in string table >2014-10-20 14:48:13,681 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) AuditLogType: USER_ADDED_DISK_PROFILE not exist in string table >2014-10-20 14:48:13,681 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) AuditLogType: USER_FAILED_TO_ADD_DISK_PROFILE not exist in string table >2014-10-20 14:48:13,681 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) AuditLogType: USER_REMOVED_DISK_PROFILE not exist in string table >2014-10-20 14:48:13,681 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) AuditLogType: USER_FAILED_TO_REMOVE_DISK_PROFILE not exist in string table >2014-10-20 14:48:13,681 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) AuditLogType: USER_UPDATED_DISK_PROFILE not exist in string table >2014-10-20 14:48:13,682 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) AuditLogType: USER_FAILED_TO_UPDATE_DISK_PROFILE not exist in string table >2014-10-20 14:48:13,682 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) AuditLogType: USER_ADDED_CPU_PROFILE not exist in string table >2014-10-20 14:48:13,682 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) AuditLogType: USER_FAILED_TO_ADD_CPU_PROFILE not exist in string table >2014-10-20 14:48:13,682 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) AuditLogType: USER_REMOVED_CPU_PROFILE not exist in string table >2014-10-20 14:48:13,682 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) AuditLogType: USER_FAILED_TO_REMOVE_CPU_PROFILE not exist in string table >2014-10-20 14:48:13,682 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) AuditLogType: USER_UPDATED_CPU_PROFILE not exist in string table >2014-10-20 14:48:13,682 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) AuditLogType: USER_FAILED_TO_UPDATE_CPU_PROFILE not exist in string table >2014-10-20 14:48:13,958 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-9) START, GetHardwareInfoVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]), log id: 6e8ce9a3 >2014-10-20 14:48:13,966 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-9) FINISH, GetHardwareInfoVDSCommand, log id: 6e8ce9a3 >2014-10-20 14:48:14,286 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-10) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 14:48:14,291 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 14:48:14,533 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-11) START, GetHardwareInfoVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, vds=Host[cyan-vdsg.qa.lab.tlv.redhat.com,3a2a45bd-4411-4428-aa45-a514d30d3ea6]), log id: 68baee3f >2014-10-20 14:48:14,543 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-10) START, GetHardwareInfoVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c]), log id: 71ba3670 >2014-10-20 14:48:14,563 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-10) FINISH, GetHardwareInfoVDSCommand, log id: 71ba3670 >2014-10-20 14:48:14,578 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-11) FINISH, GetHardwareInfoVDSCommand, log id: 68baee3f >2014-10-20 14:48:14,736 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-9) Initializing Host: alma07.qa.lab.tlv.redhat.com >2014-10-20 14:48:14,885 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-11) Initializing Host: cyan-vdsg.qa.lab.tlv.redhat.com >2014-10-20 14:48:15,188 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-9) [1285d40e] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: d5b2492d-8110-4614-9ce4-4a7a118c7942 Type: VDS >2014-10-20 14:48:15,252 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-11) [205c804a] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 3a2a45bd-4411-4428-aa45-a514d30d3ea6 Type: VDS >2014-10-20 14:48:15,280 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-10) Initializing Host: alma06.qa.lab.tlv.redhat.com >2014-10-20 14:48:15,336 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-10) [e549579] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 1c39bd88-8672-408f-a593-60d6ef8f647c Type: VDS >2014-10-20 14:48:18,534 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-2) [7e81ed2] Running command: LogoutUserCommand internal: false. >2014-10-20 14:48:18,660 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) [7e81ed2] Correlation ID: 7e81ed2, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-10-20 14:48:18,759 INFO [org.ovirt.engine.ui.frontend.server.gwt.plugin.PluginDataManager] (ajp-/127.0.0.1:8702-4) Reading UI plugin descriptor [/usr/share/ovirt-engine/ui-plugins/redhat_support_plugin_rhev.json] >2014-10-20 14:48:18,760 INFO [org.ovirt.engine.ui.frontend.server.gwt.plugin.PluginDataManager] (ajp-/127.0.0.1:8702-4) Reading UI plugin configuration [/etc/ovirt-engine/ui-plugins/redhat_support_plugin_rhev-config.json] >2014-10-20 14:48:18,837 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-4) Invalid locale found in configuration: >2014-10-20 14:48:18,837 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-4) Invalid locale found in configuration: >2014-10-20 14:48:19,530 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-31) [3a0062c3] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: f8f5eaee-8fd0-4b45-87db-62d61b03a916 Type: StoragePool >2014-10-20 14:48:19,540 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-31) [3a0062c3] Storage Pool f8f5eaee-8fd0-4b45-87db-62d61b03a916 - Updating Storage Domain a1341057-9cf8-4750-a734-656d05a5d2c5 status from Active to Unknown, reason : null >2014-10-20 14:48:19,607 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-31) [3a0062c3] Correlation ID: 3a0062c3, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center dc_35. Setting status to Non Responsive. >2014-10-20 14:48:19,858 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-31) [3a0062c3] hostFromVds::selectedVds - alma06.qa.lab.tlv.redhat.com, spmStatus Free, storage pool dc_35 >2014-10-20 14:48:19,870 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-31) [3a0062c3] Found spm host 3a2a45bd-4411-4428-aa45-a514d30d3ea6, host name: cyan-vdsg.qa.lab.tlv.redhat.com, according to spmId: 5. >2014-10-20 14:48:19,898 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-31) [3a0062c3] SpmStatus on vds 3a2a45bd-4411-4428-aa45-a514d30d3ea6: SPM >2014-10-20 14:48:19,898 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-31) [3a0062c3] Using old spm server: cyan-vdsg.qa.lab.tlv.redhat.com, no start needed >2014-10-20 14:48:19,910 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-31) [3a0062c3] Initialize Irs proxy from vds: 10.35.70.4 >2014-10-20 14:48:19,937 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-31) [3a0062c3] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Storage Pool Manager runs on Host cyan-vdsg.qa.lab.tlv.redhat.com (Address: 10.35.70.4). >2014-10-20 14:48:19,950 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to /10.35.70.4 >2014-10-20 14:48:19,979 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-15) [3a0062c3] START, SPMGetAllTasksInfoVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false), log id: 6d00345d >2014-10-20 14:48:20,184 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-15) [3a0062c3] -- executeIrsBrokerCommand: Attempting on storage pool f8f5eaee-8fd0-4b45-87db-62d61b03a916 >2014-10-20 14:48:20,235 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-15) [3a0062c3] START, HSMGetAllTasksInfoVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6), log id: 44d1cf4 >2014-10-20 14:48:20,246 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-15) [3a0062c3] FINISH, HSMGetAllTasksInfoVDSCommand, return: [], log id: 44d1cf4 >2014-10-20 14:48:20,246 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-15) [3a0062c3] FINISH, SPMGetAllTasksInfoVDSCommand, return: [], log id: 6d00345d >2014-10-20 14:48:20,247 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-7-thread-15) [3a0062c3] Discovered no tasks on Storage Pool dc_35 >2014-10-20 14:48:22,589 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (ajp-/127.0.0.1:8702-8) Running command: LoginAdminUserCommand internal: false. >2014-10-20 14:48:22,614 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-8) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-10-20 14:48:29,495 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-10) Running command: LoginUserCommand internal: false. >2014-10-20 14:48:29,526 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-10-20 14:49:00,987 INFO [org.ovirt.engine.core.bll.AddVmTemplateCommand] (ajp-/127.0.0.1:8702-2) Command 44e94001-34f8-4334-a336-fa86c26883f0 persisting async task placeholder for child command 0e8227f7-2af5-45fd-b220-78c8698264a7 >2014-10-20 14:49:01,483 INFO [org.ovirt.engine.core.bll.AddVmTemplateCommand] (ajp-/127.0.0.1:8702-2) Running command: AddVmTemplateCommand internal: false. Entities affected : ID: f8f5eaee-8fd0-4b45-87db-62d61b03a916 Type: StoragePoolAction group CREATE_TEMPLATE with role type USER >2014-10-20 14:49:01,531 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (ajp-/127.0.0.1:8702-2) START, SetVmStatusVDSCommand( vmId = 17b1e1e6-7dbe-4a3d-b556-685ca579d0e2, status = ImageLocked, exit status = Normal), log id: 4e9a5815 >2014-10-20 14:49:01,546 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (ajp-/127.0.0.1:8702-2) FINISH, SetVmStatusVDSCommand, log id: 4e9a5815 >2014-10-20 14:49:01,628 INFO [org.ovirt.engine.core.bll.CreateImageTemplateCommand] (ajp-/127.0.0.1:8702-2) [7795ba8f] Running command: CreateImageTemplateCommand internal: true. Entities affected : ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: Storage >2014-10-20 14:49:01,689 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (ajp-/127.0.0.1:8702-2) [7795ba8f] START, CopyImageVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = 40e2fdae-cc93-4242-aa96-1624d7bb788e, imageId = 79816e30-f79f-4872-a604-c7b077a7846b, dstImageGroupId = a217a5af-cb04-41e4-9252-e82aed18c789, vmId = 17b1e1e6-7dbe-4a3d-b556-685ca579d0e2, dstImageId = 0aa318ce-3e41-4359-93af-bf2228f0e5d4, imageDescription = Active VM, dstStorageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, copyVolumeType = SharedVol, volumeFormat = RAW, preallocate = Sparse, postZero = false, force = false), log id: 1f9c97d3 >2014-10-20 14:49:01,691 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (ajp-/127.0.0.1:8702-2) [7795ba8f] -- executeIrsBrokerCommand: calling 'copyImage' with two new parameters: description and UUID >2014-10-20 14:49:01,691 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (ajp-/127.0.0.1:8702-2) [7795ba8f] -- copyImage parameters: > sdUUID=a1341057-9cf8-4750-a734-656d05a5d2c5 > spUUID=f8f5eaee-8fd0-4b45-87db-62d61b03a916 > vmGUID=17b1e1e6-7dbe-4a3d-b556-685ca579d0e2 > srcImageGUID=40e2fdae-cc93-4242-aa96-1624d7bb788e > srcVolUUID=79816e30-f79f-4872-a604-c7b077a7846b > dstImageGUID=a217a5af-cb04-41e4-9252-e82aed18c789 > dstVolUUID=0aa318ce-3e41-4359-93af-bf2228f0e5d4 > descr=Active VM > dstSdUUID=a1341057-9cf8-4750-a734-656d05a5d2c5 >2014-10-20 14:49:02,009 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (ajp-/127.0.0.1:8702-2) [7795ba8f] FINISH, CopyImageVDSCommand, return: 00000000-0000-0000-0000-000000000000, log id: 1f9c97d3 >2014-10-20 14:49:02,103 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (ajp-/127.0.0.1:8702-2) [7795ba8f] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 44e94001-34f8-4334-a336-fa86c26883f0 >2014-10-20 14:49:02,104 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (ajp-/127.0.0.1:8702-2) [7795ba8f] CommandMultiAsyncTasks::AttachTask: Attaching task cc908ce6-d69a-4ca9-80b4-4b1d1e47957f to command 44e94001-34f8-4334-a336-fa86c26883f0. >2014-10-20 14:49:02,150 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (ajp-/127.0.0.1:8702-2) [7795ba8f] Adding task cc908ce6-d69a-4ca9-80b4-4b1d1e47957f (Parent Command AddVmTemplate, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-10-20 14:49:02,375 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) Correlation ID: 319010ba, Job ID: 543bbe70-3ea0-488d-b604-7051d621de3d, Call Stack: null, Custom Event ID: -1, Message: Creation of Template clean_ppc_template from VM ppc_vm was initiated by admin. >2014-10-20 14:49:02,375 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (ajp-/127.0.0.1:8702-2) BaseAsyncTask::startPollingTask: Starting to poll task cc908ce6-d69a-4ca9-80b4-4b1d1e47957f. >2014-10-20 14:49:08,572 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-7) Polling and updating Async Tasks: 1 tasks, 1 tasks to poll now >2014-10-20 14:49:08,604 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-7) SPMAsyncTask::PollTask: Polling task cc908ce6-d69a-4ca9-80b4-4b1d1e47957f (Parent Command AddVmTemplate, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status running. >2014-10-20 14:49:08,605 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-7) Finished polling Tasks, will poll again in 10 seconds. >2014-10-20 14:49:18,620 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-48) SPMAsyncTask::PollTask: Polling task cc908ce6-d69a-4ca9-80b4-4b1d1e47957f (Parent Command AddVmTemplate, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-10-20 14:49:18,639 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-48) BaseAsyncTask::onTaskEndSuccess: Task cc908ce6-d69a-4ca9-80b4-4b1d1e47957f (Parent Command AddVmTemplate, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-10-20 14:49:18,640 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-48) CommandAsyncTask::endActionIfNecessary: All tasks of command 44e94001-34f8-4334-a336-fa86c26883f0 has ended -> executing endAction >2014-10-20 14:49:18,648 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-48) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: 44e94001-34f8-4334-a336-fa86c26883f0): calling endAction . >2014-10-20 14:49:18,650 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-50) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction AddVmTemplate, executionIndex: 0 >2014-10-20 14:49:18,753 INFO [org.ovirt.engine.core.bll.AddVmTemplateCommand] (org.ovirt.thread.pool-7-thread-50) [319010ba] Ending command successfully: org.ovirt.engine.core.bll.AddVmTemplateCommand >2014-10-20 14:49:18,772 INFO [org.ovirt.engine.core.bll.CreateImageTemplateCommand] (org.ovirt.thread.pool-7-thread-50) [7795ba8f] Ending command successfully: org.ovirt.engine.core.bll.CreateImageTemplateCommand >2014-10-20 14:49:18,789 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-50) [7795ba8f] START, GetImageInfoVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = a217a5af-cb04-41e4-9252-e82aed18c789, imageId = 0aa318ce-3e41-4359-93af-bf2228f0e5d4), log id: 6ad728a0 >2014-10-20 14:49:18,841 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-50) [7795ba8f] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@a895b4b5, log id: 6ad728a0 >2014-10-20 14:49:18,868 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-50) START, SetVmStatusVDSCommand( vmId = 17b1e1e6-7dbe-4a3d-b556-685ca579d0e2, status = Down, exit status = Normal), log id: 5a14ab21 >2014-10-20 14:49:18,876 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-50) FINISH, SetVmStatusVDSCommand, log id: 5a14ab21 >2014-10-20 14:49:18,921 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-50) Correlation ID: 319010ba, Job ID: 543bbe70-3ea0-488d-b604-7051d621de3d, Call Stack: null, Custom Event ID: -1, Message: Creation of Template clean_ppc_template from VM ppc_vm has been completed. >2014-10-20 14:49:18,921 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-50) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddVmTemplate completed, handling the result. >2014-10-20 14:49:18,921 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-50) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddVmTemplate succeeded, clearing tasks. >2014-10-20 14:49:18,934 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-50) SPMAsyncTask::ClearAsyncTask: Attempting to clear task cc908ce6-d69a-4ca9-80b4-4b1d1e47957f >2014-10-20 14:49:18,935 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-50) START, SPMClearTaskVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, taskId = cc908ce6-d69a-4ca9-80b4-4b1d1e47957f), log id: 2c3b5dc9 >2014-10-20 14:49:18,939 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-50) START, HSMClearTaskVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, taskId=cc908ce6-d69a-4ca9-80b4-4b1d1e47957f), log id: 1c2b902e >2014-10-20 14:49:18,976 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-50) FINISH, HSMClearTaskVDSCommand, log id: 1c2b902e >2014-10-20 14:49:18,977 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-50) FINISH, SPMClearTaskVDSCommand, log id: 2c3b5dc9 >2014-10-20 14:49:18,993 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-50) BaseAsyncTask::removeTaskFromDB: Removed task cc908ce6-d69a-4ca9-80b4-4b1d1e47957f from DataBase >2014-10-20 14:49:18,994 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-50) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 44e94001-34f8-4334-a336-fa86c26883f0 >2014-10-20 14:49:43,168 INFO [org.ovirt.engine.core.bll.AddVmPoolWithVmsCommand] (org.ovirt.thread.pool-7-thread-30) [49eb1188] Running command: AddVmPoolWithVmsCommand internal: false. Entities affected : ID: 63702093-c04f-417c-8045-aa56334b54ee Type: VdsGroupsAction group CREATE_VM_POOL with role type USER, ID: e246188b-771c-4107-9dbf-8a61c552be0f Type: VmTemplateAction group CREATE_VM with role type USER >2014-10-20 14:49:43,384 INFO [org.ovirt.engine.core.bll.AddVmAndAttachToPoolCommand] (org.ovirt.thread.pool-7-thread-30) [5dc03e3d] Running command: AddVmAndAttachToPoolCommand internal: true. Entities affected : ID: 63702093-c04f-417c-8045-aa56334b54ee Type: VdsGroups, ID: e246188b-771c-4107-9dbf-8a61c552be0f Type: VmTemplate, ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: StorageAction group CREATE_DISK with role type USER >2014-10-20 14:49:43,402 INFO [org.ovirt.engine.core.bll.AddVmCommand] (org.ovirt.thread.pool-7-thread-30) [4bf245d1] Lock Acquired to object EngineLock [exclusiveLocks= key: test_ppc_pool-1 value: VM_NAME >, sharedLocks= key: e246188b-771c-4107-9dbf-8a61c552be0f value: TEMPLATE >key: a217a5af-cb04-41e4-9252-e82aed18c789 value: DISK >] >2014-10-20 14:49:43,446 INFO [org.ovirt.engine.core.bll.AddVmCommand] (org.ovirt.thread.pool-7-thread-30) [4bf245d1] Running command: AddVmCommand internal: true. Entities affected : ID: 63702093-c04f-417c-8045-aa56334b54ee Type: VdsGroupsAction group CREATE_VM with role type USER, ID: e246188b-771c-4107-9dbf-8a61c552be0f Type: VmTemplateAction group CREATE_VM with role type USER, ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: StorageAction group CREATE_DISK with role type USER >2014-10-20 14:49:43,523 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-30) [4bf245d1] START, SetVmStatusVDSCommand( vmId = 1f557dcf-96b2-47c2-ac4a-ef4136bfc586, status = ImageLocked, exit status = Normal), log id: 3127d4c5 >2014-10-20 14:49:43,535 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-30) [4bf245d1] FINISH, SetVmStatusVDSCommand, log id: 3127d4c5 >2014-10-20 14:49:43,741 INFO [org.ovirt.engine.core.bll.CreateSnapshotFromTemplateCommand] (org.ovirt.thread.pool-7-thread-30) [3527efff] Running command: CreateSnapshotFromTemplateCommand internal: true. Entities affected : ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: Storage >2014-10-20 14:49:43,764 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-7-thread-30) [3527efff] START, CreateSnapshotVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = daa95a9c-26c4-4324-a7ce-c86c5085bee1, imageSizeInBytes = 1073741824, volumeFormat = COW, newImageId = 23bca688-4495-47ce-bb4b-a6b4e7eb361f, newImageDescription = , imageId = 0aa318ce-3e41-4359-93af-bf2228f0e5d4, sourceImageGroupId = a217a5af-cb04-41e4-9252-e82aed18c789), log id: 117768e9 >2014-10-20 14:49:43,774 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-7-thread-30) [3527efff] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-10-20 14:49:43,919 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-7-thread-30) [3527efff] FINISH, CreateSnapshotVDSCommand, return: 23bca688-4495-47ce-bb4b-a6b4e7eb361f, log id: 117768e9 >2014-10-20 14:49:43,921 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-30) [3527efff] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 7e4ce1b9-80c1-411f-94bf-398e5c1d87af >2014-10-20 14:49:43,921 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-7-thread-30) [3527efff] CommandMultiAsyncTasks::AttachTask: Attaching task 695a2688-e586-4360-a02b-66403983f4ab to command 7e4ce1b9-80c1-411f-94bf-398e5c1d87af. >2014-10-20 14:49:43,965 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-7-thread-30) [3527efff] Adding task 695a2688-e586-4360-a02b-66403983f4ab (Parent Command AddVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-10-20 14:49:44,222 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-30) [3527efff] Correlation ID: 4bf245d1, Call Stack: null, Custom Event ID: -1, Message: VM test_ppc_pool-1 creation was initiated by admin. >2014-10-20 14:49:44,222 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-30) [3527efff] BaseAsyncTask::startPollingTask: Starting to poll task 695a2688-e586-4360-a02b-66403983f4ab. >2014-10-20 14:49:44,367 INFO [org.ovirt.engine.core.bll.AddVmToPoolCommand] (org.ovirt.thread.pool-7-thread-30) [94b428f] Running command: AddVmToPoolCommand internal: true. Entities affected : ID: 0c2de4aa-2d90-4c80-812d-625763aa8c12 Type: VmPool >2014-10-20 14:49:44,466 INFO [org.ovirt.engine.core.bll.AddVmAndAttachToPoolCommand] (org.ovirt.thread.pool-7-thread-30) [110b0a72] Running command: AddVmAndAttachToPoolCommand internal: true. Entities affected : ID: 63702093-c04f-417c-8045-aa56334b54ee Type: VdsGroups, ID: e246188b-771c-4107-9dbf-8a61c552be0f Type: VmTemplate, ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: StorageAction group CREATE_DISK with role type USER >2014-10-20 14:49:44,480 INFO [org.ovirt.engine.core.bll.AddVmCommand] (org.ovirt.thread.pool-7-thread-30) [44a55885] Lock Acquired to object EngineLock [exclusiveLocks= key: test_ppc_pool-2 value: VM_NAME >, sharedLocks= key: e246188b-771c-4107-9dbf-8a61c552be0f value: TEMPLATE >key: a217a5af-cb04-41e4-9252-e82aed18c789 value: DISK >] >2014-10-20 14:49:44,576 INFO [org.ovirt.engine.core.bll.AddVmCommand] (org.ovirt.thread.pool-7-thread-30) [44a55885] Running command: AddVmCommand internal: true. Entities affected : ID: 63702093-c04f-417c-8045-aa56334b54ee Type: VdsGroupsAction group CREATE_VM with role type USER, ID: e246188b-771c-4107-9dbf-8a61c552be0f Type: VmTemplateAction group CREATE_VM with role type USER, ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: StorageAction group CREATE_DISK with role type USER >2014-10-20 14:49:44,637 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-30) [44a55885] START, SetVmStatusVDSCommand( vmId = e7b180d9-f84e-40f9-9d25-6a62bb713699, status = ImageLocked, exit status = Normal), log id: 72f05a3c >2014-10-20 14:49:44,646 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-30) [44a55885] FINISH, SetVmStatusVDSCommand, log id: 72f05a3c >2014-10-20 14:49:44,729 INFO [org.ovirt.engine.core.bll.CreateSnapshotFromTemplateCommand] (org.ovirt.thread.pool-7-thread-30) [36fa0073] Running command: CreateSnapshotFromTemplateCommand internal: true. Entities affected : ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: Storage >2014-10-20 14:49:44,735 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-7-thread-30) [36fa0073] START, CreateSnapshotVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = 45206a2d-81c8-43fe-a2f7-8ffee8f6c96c, imageSizeInBytes = 1073741824, volumeFormat = COW, newImageId = fb4eb631-31c1-4230-b593-8755deb71a3f, newImageDescription = , imageId = 0aa318ce-3e41-4359-93af-bf2228f0e5d4, sourceImageGroupId = a217a5af-cb04-41e4-9252-e82aed18c789), log id: 26a30e71 >2014-10-20 14:49:44,736 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-7-thread-30) [36fa0073] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-10-20 14:49:44,872 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-7-thread-30) [36fa0073] FINISH, CreateSnapshotVDSCommand, return: fb4eb631-31c1-4230-b593-8755deb71a3f, log id: 26a30e71 >2014-10-20 14:49:44,873 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-30) [36fa0073] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 39f02f4c-dd4f-489d-be28-88d323bf9cfc >2014-10-20 14:49:44,873 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-7-thread-30) [36fa0073] CommandMultiAsyncTasks::AttachTask: Attaching task 3faf7225-fb58-49e4-bfd2-88d12529da00 to command 39f02f4c-dd4f-489d-be28-88d323bf9cfc. >2014-10-20 14:49:44,927 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-7-thread-30) [36fa0073] Adding task 3faf7225-fb58-49e4-bfd2-88d12529da00 (Parent Command AddVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-10-20 14:49:45,237 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-30) [36fa0073] Correlation ID: 44a55885, Call Stack: null, Custom Event ID: -1, Message: VM test_ppc_pool-2 creation was initiated by admin. >2014-10-20 14:49:45,237 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-30) [36fa0073] BaseAsyncTask::startPollingTask: Starting to poll task 3faf7225-fb58-49e4-bfd2-88d12529da00. >2014-10-20 14:49:45,296 INFO [org.ovirt.engine.core.bll.AddVmToPoolCommand] (org.ovirt.thread.pool-7-thread-30) [1259bb82] Running command: AddVmToPoolCommand internal: true. Entities affected : ID: 0c2de4aa-2d90-4c80-812d-625763aa8c12 Type: VmPool >2014-10-20 14:49:45,345 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-30) [1259bb82] Correlation ID: 49eb1188, Job ID: 3c05f37e-16e9-4251-87f7-2a8c79b8606c, Call Stack: null, Custom Event ID: -1, Message: VM Pool test_ppc_pool (containing 2 VMs) was created by admin. >2014-10-20 14:49:48,657 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-61) Polling and updating Async Tasks: 3 tasks, 2 tasks to poll now >2014-10-20 14:49:48,681 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-61) SPMAsyncTask::PollTask: Polling task 695a2688-e586-4360-a02b-66403983f4ab (Parent Command AddVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-10-20 14:49:48,682 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-61) BaseAsyncTask::onTaskEndSuccess: Task 695a2688-e586-4360-a02b-66403983f4ab (Parent Command AddVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-10-20 14:49:48,682 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-61) CommandAsyncTask::endActionIfNecessary: All tasks of command 7e4ce1b9-80c1-411f-94bf-398e5c1d87af has ended -> executing endAction >2014-10-20 14:49:48,683 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-61) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: 7e4ce1b9-80c1-411f-94bf-398e5c1d87af): calling endAction . >2014-10-20 14:49:48,683 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-61) SPMAsyncTask::PollTask: Polling task 3faf7225-fb58-49e4-bfd2-88d12529da00 (Parent Command AddVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-10-20 14:49:48,683 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-61) BaseAsyncTask::onTaskEndSuccess: Task 3faf7225-fb58-49e4-bfd2-88d12529da00 (Parent Command AddVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-10-20 14:49:48,684 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-61) CommandAsyncTask::endActionIfNecessary: All tasks of command 39f02f4c-dd4f-489d-be28-88d323bf9cfc has ended -> executing endAction >2014-10-20 14:49:48,684 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-61) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: 39f02f4c-dd4f-489d-be28-88d323bf9cfc): calling endAction . >2014-10-20 14:49:48,687 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-40) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction AddVm, executionIndex: 0 >2014-10-20 14:49:48,695 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-42) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction AddVm, executionIndex: 0 >2014-10-20 14:49:48,718 INFO [org.ovirt.engine.core.bll.AddVmCommand] (org.ovirt.thread.pool-7-thread-40) [4bf245d1] Ending command successfully: org.ovirt.engine.core.bll.AddVmCommand >2014-10-20 14:49:48,778 INFO [org.ovirt.engine.core.bll.AddVmCommand] (org.ovirt.thread.pool-7-thread-42) [44a55885] Ending command successfully: org.ovirt.engine.core.bll.AddVmCommand >2014-10-20 14:49:48,823 INFO [org.ovirt.engine.core.bll.CreateSnapshotFromTemplateCommand] (org.ovirt.thread.pool-7-thread-40) [3527efff] Ending command successfully: org.ovirt.engine.core.bll.CreateSnapshotFromTemplateCommand >2014-10-20 14:49:48,837 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-40) [3527efff] START, GetImageInfoVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = daa95a9c-26c4-4324-a7ce-c86c5085bee1, imageId = 23bca688-4495-47ce-bb4b-a6b4e7eb361f), log id: 5838037f >2014-10-20 14:49:48,838 INFO [org.ovirt.engine.core.bll.CreateSnapshotFromTemplateCommand] (org.ovirt.thread.pool-7-thread-42) [36fa0073] Ending command successfully: org.ovirt.engine.core.bll.CreateSnapshotFromTemplateCommand >2014-10-20 14:49:48,842 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-42) [36fa0073] START, GetImageInfoVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = 45206a2d-81c8-43fe-a2f7-8ffee8f6c96c, imageId = fb4eb631-31c1-4230-b593-8755deb71a3f), log id: 2b07db65 >2014-10-20 14:49:48,873 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-40) [3527efff] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@c53ceee, log id: 5838037f >2014-10-20 14:49:48,908 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-40) START, SetVmStatusVDSCommand( vmId = 1f557dcf-96b2-47c2-ac4a-ef4136bfc586, status = Down, exit status = Normal), log id: 16f77c33 >2014-10-20 14:49:48,913 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-40) FINISH, SetVmStatusVDSCommand, log id: 16f77c33 >2014-10-20 14:49:48,915 INFO [org.ovirt.engine.core.bll.AddVmCommand] (org.ovirt.thread.pool-7-thread-40) Lock freed to object EngineLock [exclusiveLocks= key: test_ppc_pool-1 value: VM_NAME >, sharedLocks= key: e246188b-771c-4107-9dbf-8a61c552be0f value: TEMPLATE >key: a217a5af-cb04-41e4-9252-e82aed18c789 value: DISK >] >2014-10-20 14:49:48,920 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-42) [36fa0073] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@d04824, log id: 2b07db65 >2014-10-20 14:49:48,935 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-42) START, SetVmStatusVDSCommand( vmId = e7b180d9-f84e-40f9-9d25-6a62bb713699, status = Down, exit status = Normal), log id: 204ef50f >2014-10-20 14:49:48,936 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-40) Correlation ID: 4bf245d1, Call Stack: null, Custom Event ID: -1, Message: VM test_ppc_pool-1 creation has been completed. >2014-10-20 14:49:48,936 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-40) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddVm completed, handling the result. >2014-10-20 14:49:48,936 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-40) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddVm succeeded, clearing tasks. >2014-10-20 14:49:48,937 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-40) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 695a2688-e586-4360-a02b-66403983f4ab >2014-10-20 14:49:48,937 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-40) START, SPMClearTaskVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, taskId = 695a2688-e586-4360-a02b-66403983f4ab), log id: 55b66205 >2014-10-20 14:49:48,939 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-40) START, HSMClearTaskVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, taskId=695a2688-e586-4360-a02b-66403983f4ab), log id: 38dd6239 >2014-10-20 14:49:48,943 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-42) FINISH, SetVmStatusVDSCommand, log id: 204ef50f >2014-10-20 14:49:48,945 INFO [org.ovirt.engine.core.bll.AddVmCommand] (org.ovirt.thread.pool-7-thread-42) Lock freed to object EngineLock [exclusiveLocks= key: test_ppc_pool-2 value: VM_NAME >, sharedLocks= key: e246188b-771c-4107-9dbf-8a61c552be0f value: TEMPLATE >key: a217a5af-cb04-41e4-9252-e82aed18c789 value: DISK >] >2014-10-20 14:49:48,950 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-42) Correlation ID: 44a55885, Call Stack: null, Custom Event ID: -1, Message: VM test_ppc_pool-2 creation has been completed. >2014-10-20 14:49:48,950 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-42) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddVm completed, handling the result. >2014-10-20 14:49:48,950 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-42) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddVm succeeded, clearing tasks. >2014-10-20 14:49:48,951 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-42) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 3faf7225-fb58-49e4-bfd2-88d12529da00 >2014-10-20 14:49:48,951 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-42) START, SPMClearTaskVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, taskId = 3faf7225-fb58-49e4-bfd2-88d12529da00), log id: 675a6adf >2014-10-20 14:49:48,979 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-40) FINISH, HSMClearTaskVDSCommand, log id: 38dd6239 >2014-10-20 14:49:48,979 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-40) FINISH, SPMClearTaskVDSCommand, log id: 55b66205 >2014-10-20 14:49:48,993 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-42) START, HSMClearTaskVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, taskId=3faf7225-fb58-49e4-bfd2-88d12529da00), log id: 2ce5d70a >2014-10-20 14:49:48,994 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-40) BaseAsyncTask::removeTaskFromDB: Removed task 695a2688-e586-4360-a02b-66403983f4ab from DataBase >2014-10-20 14:49:48,995 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-40) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 7e4ce1b9-80c1-411f-94bf-398e5c1d87af >2014-10-20 14:49:49,038 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-42) FINISH, HSMClearTaskVDSCommand, log id: 2ce5d70a >2014-10-20 14:49:49,039 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-42) FINISH, SPMClearTaskVDSCommand, log id: 675a6adf >2014-10-20 14:49:49,045 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-42) BaseAsyncTask::removeTaskFromDB: Removed task 3faf7225-fb58-49e4-bfd2-88d12529da00 from DataBase >2014-10-20 14:49:49,045 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-42) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 39f02f4c-dd4f-489d-be28-88d323bf9cfc >2014-10-20 14:50:31,169 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-4) Running command: LoginUserCommand internal: false. >2014-10-20 14:50:31,173 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-10-20 14:50:31,340 INFO [org.ovirt.engine.core.bll.aaa.LogoutBySessionCommand] (ajp-/127.0.0.1:8702-4) [d3cb46d] Running command: LogoutBySessionCommand internal: false. >2014-10-20 14:50:31,354 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-4) [3b475ebf] Running command: LogoutUserCommand internal: false. >2014-10-20 14:50:31,362 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [3b475ebf] Correlation ID: 3b475ebf, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-10-20 14:50:32,209 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-5) Running command: LoginUserCommand internal: false. >2014-10-20 14:50:38,509 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-46) Setting new tasks map. The map contains now 2 tasks >2014-10-20 14:50:38,776 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-1) Running command: LoginUserCommand internal: false. >2014-10-20 14:50:38,779 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-10-20 14:50:39,499 INFO [org.ovirt.engine.core.bll.aaa.LogoutBySessionCommand] (ajp-/127.0.0.1:8702-1) [57f9d35b] Running command: LogoutBySessionCommand internal: false. >2014-10-20 14:50:39,511 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-1) [706e2d1d] Running command: LogoutUserCommand internal: false. >2014-10-20 14:50:39,515 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [706e2d1d] Correlation ID: 706e2d1d, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-10-20 14:51:08,511 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-56) Setting new tasks map. The map contains now 0 tasks >2014-10-20 14:51:08,511 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-56) Cleared all tasks of pool f8f5eaee-8fd0-4b45-87db-62d61b03a916. >2014-10-20 14:51:11,555 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-8) Running command: LoginUserCommand internal: false. >2014-10-20 14:51:11,567 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-8) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-10-20 14:51:11,785 INFO [org.ovirt.engine.core.bll.UpdateVmPoolWithVmsCommand] (ajp-/127.0.0.1:8702-8) [1fe4f97a] Running command: UpdateVmPoolWithVmsCommand internal: false. Entities affected : ID: 0c2de4aa-2d90-4c80-812d-625763aa8c12 Type: VmPoolAction group EDIT_VM_POOL_CONFIGURATION with role type USER >2014-10-20 14:51:11,863 WARN [org.ovirt.engine.core.bll.AddVmAndAttachToPoolCommand] (ajp-/127.0.0.1:8702-8) [659ddf5] CanDoAction of action AddVmAndAttachToPool failed. Reasons:VAR__ACTION__ADD,VAR__TYPE__VM,ACTION_TYPE_FAILED_ILLEGAL_OS_TYPE_IS_NOT_SUPPORTED_BY_ARCHITECTURE_TYPE >2014-10-20 14:51:11,949 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-8) [659ddf5] Correlation ID: 1fe4f97a, Job ID: 166e3eed-675e-4069-8ba4-19132ebb243d, Call Stack: null, Custom Event ID: -1, Message: Failed to update VM Pool test_ppc_pool(User: admin). >2014-10-20 14:51:11,958 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ajp-/127.0.0.1:8702-8) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table. >2014-10-20 14:51:11,967 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (ajp-/127.0.0.1:8702-8) Operation Failed: [Cannot edit VM-Pool. Selected operating system is not supported by the architecture.] >2014-10-20 14:51:12,533 INFO [org.ovirt.engine.core.bll.aaa.LogoutBySessionCommand] (ajp-/127.0.0.1:8702-8) [68492c1d] Running command: LogoutBySessionCommand internal: false. >2014-10-20 14:51:12,535 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-8) [23ae43c6] Running command: LogoutUserCommand internal: false. >2014-10-20 14:51:12,538 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-8) [23ae43c6] Correlation ID: 23ae43c6, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-10-20 14:51:32,211 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-11) Running command: LoginUserCommand internal: false. >2014-10-20 14:51:32,216 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-10-20 14:59:07,052 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/usr/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf". >2014-10-20 14:59:07,052 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) The file "/etc/ovirt-engine/engine.conf" doesn't exist or isn't readable. Will return an empty set of properties. >2014-10-20 14:59:07,053 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/10-rhevm-doc.conf". >2014-10-20 14:59:07,053 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-database.conf". >2014-10-20 14:59:07,054 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf". >2014-10-20 14:59:07,054 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf". >2014-10-20 14:59:07,055 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf". >2014-10-20 14:59:07,055 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/20-branding-rhev.conf". >2014-10-20 14:59:07,055 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/50-redhat-support-plugin.conf". >2014-10-20 14:59:07,056 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_AJP_ENABLED" is "true". >2014-10-20 14:59:07,056 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_AJP_PORT" is "8702". >2014-10-20 14:59:07,057 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_APPS" is "engine.ear restapi.war legacy_restapi.war "/usr/share/ovirt-engine/branding/rhev-2.brand/applications/rhev.ear" redhat-support-plugin-rhev.ear". >2014-10-20 14:59:07,057 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_CACHE" is "/var/cache/ovirt-engine". >2014-10-20 14:59:07,057 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_CHECK_INTERVAL" is "1000". >2014-10-20 14:59:07,057 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_CONNECTION_TIMEOUT" is "300000". >2014-10-20 14:59:07,058 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_DATABASE" is "engine". >2014-10-20 14:59:07,058 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_DRIVER" is "org.postgresql.Driver". >2014-10-20 14:59:07,058 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_HOST" is "localhost". >2014-10-20 14:59:07,058 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_MAX_CONNECTIONS" is "100". >2014-10-20 14:59:07,058 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_MIN_CONNECTIONS" is "1". >2014-10-20 14:59:07,059 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_PASSWORD" is "***". >2014-10-20 14:59:07,059 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_PORT" is "5432". >2014-10-20 14:59:07,059 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_SECURED" is "False". >2014-10-20 14:59:07,059 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_SECURED_VALIDATION" is "False". >2014-10-20 14:59:07,070 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_URL" is "jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory". >2014-10-20 14:59:07,070 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_USER" is "engine". >2014-10-20 14:59:07,070 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DEBUG_ADDRESS" is "". >2014-10-20 14:59:07,070 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DOC" is "/usr/share/doc/ovirt-engine". >2014-10-20 14:59:07,071 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_ETC" is "/etc/ovirt-engine". >2014-10-20 14:59:07,071 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_EXTENSION_PATH" is "/usr/share/ovirt-engine/extensions.d:/etc/ovirt-engine/extensions.d". >2014-10-20 14:59:07,071 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_FQDN" is "dhcp163-98.scl.lab.tlv.redhat.com". >2014-10-20 14:59:07,071 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_GROUP" is "ovirt". >2014-10-20 14:59:07,072 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HEAP_MAX" is "1g". >2014-10-20 14:59:07,072 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HEAP_MIN" is "1g". >2014-10-20 14:59:07,072 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HTTPS_ENABLED" is "false". >2014-10-20 14:59:07,072 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HTTPS_PORT" is "None". >2014-10-20 14:59:07,072 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HTTPS_PROTOCOLS" is "SSLv3,TLSv1,TLSv1.1,TLSv1.2". >2014-10-20 14:59:07,073 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HTTP_ENABLED" is "false". >2014-10-20 14:59:07,073 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HTTP_PORT" is "None". >2014-10-20 14:59:07,073 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_JAVA_MODULEPATH" is "/usr/share/ovirt-engine/modules". >2014-10-20 14:59:07,073 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_JVM_ARGS" is " -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/var/log/ovirt-engine/dump"". >2014-10-20 14:59:07,073 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_LOG" is "/var/log/ovirt-engine". >2014-10-20 14:59:07,074 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_LOG_TO_CONSOLE" is "false". >2014-10-20 14:59:07,074 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_MANUAL" is "/usr/share/doc/rhevm-doc/manual". >2014-10-20 14:59:07,074 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PERM_MAX" is "256m". >2014-10-20 14:59:07,074 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PERM_MIN" is "256m". >2014-10-20 14:59:07,075 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI" is "/etc/pki/ovirt-engine". >2014-10-20 14:59:07,075 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_CA" is "/etc/pki/ovirt-engine/ca.pem". >2014-10-20 14:59:07,075 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_ENGINE_CERT" is "/etc/pki/ovirt-engine/certs/engine.cer". >2014-10-20 14:59:07,075 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_ENGINE_STORE" is "/etc/pki/ovirt-engine/keys/engine.p12". >2014-10-20 14:59:07,075 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_ENGINE_STORE_ALIAS" is "1". >2014-10-20 14:59:07,076 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_ENGINE_STORE_PASSWORD" is "***". >2014-10-20 14:59:07,076 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_TRUST_STORE" is "/etc/pki/ovirt-engine/.truststore". >2014-10-20 14:59:07,076 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_TRUST_STORE_PASSWORD" is "***". >2014-10-20 14:59:07,076 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PROPERTIES" is " jsse.enableSNIExtension=false "java.security.krb5.conf=/etc/ovirt-engine/krb5.conf"". >2014-10-20 14:59:07,077 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PROXY_ENABLED" is "true". >2014-10-20 14:59:07,077 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PROXY_HTTPS_PORT" is "443". >2014-10-20 14:59:07,077 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PROXY_HTTP_PORT" is "80". >2014-10-20 14:59:07,077 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_REPORTS_BASE_URL" is "". >2014-10-20 14:59:07,077 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_REPORTS_DASHBOARD_URL" is "/flow.html?viewAsDashboardFrame=true". >2014-10-20 14:59:07,078 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_REPORTS_NOT_INSTALLED_URL" is "/ovirt-engine/ReportsNotInstalled.html". >2014-10-20 14:59:07,078 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_REPORTS_PROXY_URL" is "/ovirt/reports-interface". >2014-10-20 14:59:07,078 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_REPORTS_READ_TIMEOUT" is "". >2014-10-20 14:59:07,078 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_REPORTS_RIGHTCLICK_URL" is "". >2014-10-20 14:59:07,078 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_REPORTS_VERIFY_CHAIN" is "true". >2014-10-20 14:59:07,079 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_REPORTS_VERIFY_HOST" is "true". >2014-10-20 14:59:07,079 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_STOP_INTERVAL" is "1". >2014-10-20 14:59:07,079 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_STOP_TIME" is "10". >2014-10-20 14:59:07,079 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_TMP" is "/var/tmp/ovirt-engine". >2014-10-20 14:59:07,156 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_UP_MARK" is "/var/lib/ovirt-engine/engine.up". >2014-10-20 14:59:07,156 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_URI" is "/ovirt-engine". >2014-10-20 14:59:07,156 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_USER" is "ovirt". >2014-10-20 14:59:07,157 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_USR" is "/usr/share/ovirt-engine". >2014-10-20 14:59:07,157 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_VAR" is "/var/lib/ovirt-engine". >2014-10-20 14:59:07,157 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_VERBOSE_GC" is "false". >2014-10-20 14:59:07,157 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "JBOSS_HOME" is "/usr/share/jbossas". >2014-10-20 14:59:07,157 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "JBOSS_RUNTIME" is "/var/lib/ovirt-engine/jboss_runtime". >2014-10-20 14:59:07,157 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "SENSITIVE_KEYS" is ",ENGINE_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD". >2014-10-20 14:59:19,538 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 41) Start initializing Backend >2014-10-20 14:59:20,815 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (ServerService Thread Pool -- 41) Loaded file /etc/ovirt-engine/osinfo.conf.d/00-defaults.properties >2014-10-20 14:59:21,253 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 41) Running ovirt-engine 3.5.0-0.14.beta.el6ev >2014-10-20 14:59:21,254 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ServerService Thread Pool -- 41) Start initializing dictionaries >2014-10-20 14:59:21,258 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ServerService Thread Pool -- 41) Finished initializing dictionaries >2014-10-20 14:59:21,258 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 41) Start initializing AuditLogCleanupManager >2014-10-20 14:59:21,258 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 41) Setting audit cleanup manager to run at: 35 35 3 * * ? >2014-10-20 14:59:21,350 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 41) Finished initializing AuditLogCleanupManager >2014-10-20 14:59:21,350 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 41) Start initializing CommandEntityCleanupManager >2014-10-20 14:59:21,351 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 41) Setting command entity cleanup manager to run at: 35 35 3 * * ? >2014-10-20 14:59:21,355 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 41) Finished initializing CommandEntityCleanupManager >2014-10-20 14:59:21,357 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 41) Start initializing TagsDirector >2014-10-20 14:59:21,375 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 41) Tag root added to tree >2014-10-20 14:59:21,397 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 41) Finished initializing TagsDirector >2014-10-20 14:59:21,398 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (ServerService Thread Pool -- 41) Start initializing IsoDomainListSyncronizer >2014-10-20 14:59:21,419 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (ServerService Thread Pool -- 41) Finished initializing IsoDomainListSyncronizer >2014-10-20 14:59:21,593 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 41) Completed initializing handlers >2014-10-20 14:59:21,664 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 41) Start initializing ErrorTranslatorImpl >2014-10-20 14:59:21,743 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 41) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table. >2014-10-20 14:59:21,745 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 41) Finished initializing ErrorTranslatorImpl >2014-10-20 14:59:21,745 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 41) Start initializing ErrorTranslatorImpl >2014-10-20 14:59:21,746 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 41) Finished initializing ErrorTranslatorImpl >2014-10-20 14:59:21,768 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 41) Mark incomplete jobs as UNKNOWN >2014-10-20 14:59:21,995 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ServerService Thread Pool -- 41) Start initializing JobRepositoryCleanupManager >2014-10-20 14:59:21,996 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ServerService Thread Pool -- 41) Finished initializing JobRepositoryCleanupManager >2014-10-20 14:59:21,996 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ServerService Thread Pool -- 41) Start initializing AutoRecoveryManager >2014-10-20 14:59:22,004 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ServerService Thread Pool -- 41) Finished initializing AutoRecoveryManager >2014-10-20 14:59:22,006 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ServerService Thread Pool -- 41) Start initializing ExecutionMessageDirector >2014-10-20 14:59:22,009 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ServerService Thread Pool -- 41) Finished initializing ExecutionMessageDirector >2014-10-20 14:59:22,214 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 41) Loading extension 'builtin-authn-internal' >2014-10-20 14:59:22,215 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 41) Extension 'builtin-authn-internal' loaded >2014-10-20 14:59:22,229 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 41) Loading extension 'internal' >2014-10-20 14:59:22,232 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 41) Extension 'internal' loaded >2014-10-20 14:59:22,261 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 41) Loading extension 'builtin-authn-qa.lab.tlv.redhat.com' >2014-10-20 14:59:22,284 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 41) Extension 'builtin-authn-qa.lab.tlv.redhat.com' loaded >2014-10-20 14:59:22,286 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 41) Loading extension 'qa.lab.tlv.redhat.com' >2014-10-20 14:59:22,287 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 41) Extension 'qa.lab.tlv.redhat.com' loaded >2014-10-20 14:59:22,293 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 41) Initializing extension 'qa.lab.tlv.redhat.com' >2014-10-20 14:59:22,293 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 41) Extension 'qa.lab.tlv.redhat.com' initialized >2014-10-20 14:59:22,294 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 41) Initializing extension 'builtin-authn-internal' >2014-10-20 14:59:22,294 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 41) Extension 'builtin-authn-internal' initialized >2014-10-20 14:59:22,295 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 41) Initializing extension 'builtin-authn-qa.lab.tlv.redhat.com' >2014-10-20 14:59:22,295 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 41) Extension 'builtin-authn-qa.lab.tlv.redhat.com' initialized >2014-10-20 14:59:22,296 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 41) Initializing extension 'internal' >2014-10-20 14:59:22,296 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 41) Extension 'internal' initialized >2014-10-20 14:59:22,296 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 41) Start of enabled extensions list >2014-10-20 14:59:22,297 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 41) Instance name: 'qa.lab.tlv.redhat.com', Extension name: 'Kerberos/Ldap Authz (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-10-20 14:59:22,297 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 41) Instance name: 'builtin-authn-internal', Extension name: 'Internal Authn (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-10-20 14:59:22,297 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 41) Instance name: 'builtin-authn-qa.lab.tlv.redhat.com', Extension name: 'Kerberos/Ldap Authn (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-10-20 14:59:22,297 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 41) Instance name: 'internal', Extension name: 'Internal Authz (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-10-20 14:59:22,297 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 41) End of enabled extensions list >2014-10-20 14:59:22,790 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (ServerService Thread Pool -- 55) Start initializing DbUserCacheManager >2014-10-20 14:59:22,844 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (ServerService Thread Pool -- 55) Finished initializing DbUserCacheManager >2014-10-20 14:59:22,887 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (ServerService Thread Pool -- 55) Initialization of AsyncTaskManager completed successfully. >2014-10-20 14:59:22,890 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 55) Start initializing ResourceManager >2014-10-20 14:59:23,198 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 55) Entered VdsManager constructor >2014-10-20 14:59:23,301 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 55) Initialize vdsBroker (alma06.qa.lab.tlv.redhat.com,54,321) >2014-10-20 14:59:23,394 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-2) Deactivating non existing principal admin::internal >2014-10-20 14:59:23,803 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 55) VDS 1c39bd88-8672-408f-a593-60d6ef8f647c was added to the Resource Manager >2014-10-20 14:59:23,804 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 55) Entered VdsManager constructor >2014-10-20 14:59:23,836 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 55) Initialize vdsBroker (alma07.qa.lab.tlv.redhat.com,54,321) >2014-10-20 14:59:23,871 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 55) VDS d5b2492d-8110-4614-9ce4-4a7a118c7942 was added to the Resource Manager >2014-10-20 14:59:23,871 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 55) Entered VdsManager constructor >2014-10-20 14:59:23,901 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 55) Initialize vdsBroker (rose05.qa.lab.tlv.redhat.com,54,321) >2014-10-20 14:59:23,934 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 55) VDS 45869c56-4fbd-466b-977a-81604a392b78 was added to the Resource Manager >2014-10-20 14:59:23,934 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 55) Entered VdsManager constructor >2014-10-20 14:59:23,960 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 55) Initialize vdsBroker (master-vds10.qa.lab.tlv.redhat.com,54,321) >2014-10-20 14:59:23,987 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 55) VDS 51fdbad1-2bd5-47d4-aa62-4054e71fbbbf was added to the Resource Manager >2014-10-20 14:59:23,987 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 55) Entered VdsManager constructor >2014-10-20 14:59:24,013 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 55) Initialize vdsBroker (10.35.70.4,54,321) >2014-10-20 14:59:24,040 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 55) VDS 3a2a45bd-4411-4428-aa45-a514d30d3ea6 was added to the Resource Manager >2014-10-20 14:59:24,089 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 55) Finished initializing ResourceManager >2014-10-20 14:59:24,096 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (ServerService Thread Pool -- 55) Initialization of OvfDataUpdater completed successfully. >2014-10-20 14:59:24,329 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 55) Start scheduling to enable vds load balancer >2014-10-20 14:59:24,330 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 55) Finished scheduling to enable vds load balancer >2014-10-20 14:59:24,330 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 55) Start HA Reservation check >2014-10-20 14:59:24,331 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 55) Finished HA Reservation check >2014-10-20 14:59:24,348 INFO [org.ovirt.engine.core.bll.network.MacPoolManagerRanges] (org.ovirt.thread.pool-7-thread-1) Start initializing MacPoolManagerRanges >2014-10-20 14:59:24,379 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 55) Init VM custom properties utilities >2014-10-20 14:59:24,380 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 55) Init device custom properties utilities >2014-10-20 14:59:24,385 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 55) Initializing Scheduling manager >2014-10-20 14:59:24,418 INFO [org.ovirt.engine.core.bll.network.MacPoolManagerRanges] (org.ovirt.thread.pool-7-thread-1) Finished initializing. Available MACs in pool: 246 >2014-10-20 14:59:24,474 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 55) Starting external scheduler dicovery thread >2014-10-20 14:59:24,488 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 55) Initialized Scheduling manager >2014-10-20 14:59:24,488 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (ServerService Thread Pool -- 55) Initializing DWH Heart Beat >2014-10-20 14:59:24,489 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (ServerService Thread Pool -- 55) DWH Heart Beat initialized >2014-10-20 14:59:24,739 INFO [org.ovirt.engine.core.bll.scheduling.external.ExternalSchedulerDiscoveryThread] (Thread-71) PolicyUnits updated >2014-10-20 14:59:25,707 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-3) [783e0dc7] Running command: LogoutUserCommand internal: false. >2014-10-20 14:59:25,708 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-5) [30b5819d] Running command: LogoutUserCommand internal: false. >2014-10-20 14:59:25,708 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-2) [414f8566] Running command: LogoutUserCommand internal: false. >2014-10-20 14:59:25,709 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-7) [1d302cac] Running command: LogoutUserCommand internal: false. >2014-10-20 14:59:25,709 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-4) [27c867b8] Running command: LogoutUserCommand internal: false. >2014-10-20 14:59:25,709 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-6) [6e44a481] Running command: LogoutUserCommand internal: false. >2014-10-20 14:59:25,834 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [30b5819d] AuditLogType: UNASSIGNED not exist in string table >2014-10-20 14:59:25,834 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [30b5819d] AuditLogType: VDS_HIGH_NETWORK_USE not exist in string table >2014-10-20 14:59:25,835 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [30b5819d] AuditLogType: USER_FAILED_REMOVE_VM not exist in string table >2014-10-20 14:59:25,835 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [30b5819d] AuditLogType: USER_RUN_UNLOCK_ENTITY_SCRIPT not exist in string table >2014-10-20 14:59:25,836 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [30b5819d] AuditLogType: VDS_NETWORK_MTU_DIFFER_FROM_LOGICAL_NETWORK not exist in string table >2014-10-20 14:59:25,837 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [30b5819d] AuditLogType: STORAGE_ACTIVATE_ASYNC not exist in string table >2014-10-20 14:59:25,837 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [30b5819d] AuditLogType: USER_ADDED_DISK_PROFILE not exist in string table >2014-10-20 14:59:25,837 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [30b5819d] AuditLogType: USER_FAILED_TO_ADD_DISK_PROFILE not exist in string table >2014-10-20 14:59:25,837 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [30b5819d] AuditLogType: USER_REMOVED_DISK_PROFILE not exist in string table >2014-10-20 14:59:25,838 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [30b5819d] AuditLogType: USER_FAILED_TO_REMOVE_DISK_PROFILE not exist in string table >2014-10-20 14:59:25,838 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [30b5819d] AuditLogType: USER_UPDATED_DISK_PROFILE not exist in string table >2014-10-20 14:59:25,838 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [30b5819d] AuditLogType: USER_FAILED_TO_UPDATE_DISK_PROFILE not exist in string table >2014-10-20 14:59:25,838 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [30b5819d] AuditLogType: USER_ADDED_CPU_PROFILE not exist in string table >2014-10-20 14:59:25,838 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [30b5819d] AuditLogType: USER_FAILED_TO_ADD_CPU_PROFILE not exist in string table >2014-10-20 14:59:25,838 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [30b5819d] AuditLogType: USER_REMOVED_CPU_PROFILE not exist in string table >2014-10-20 14:59:25,839 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [30b5819d] AuditLogType: USER_FAILED_TO_REMOVE_CPU_PROFILE not exist in string table >2014-10-20 14:59:25,839 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [30b5819d] AuditLogType: USER_UPDATED_CPU_PROFILE not exist in string table >2014-10-20 14:59:25,839 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [30b5819d] AuditLogType: USER_FAILED_TO_UPDATE_CPU_PROFILE not exist in string table >2014-10-20 14:59:25,939 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [30b5819d] Correlation ID: 30b5819d, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-10-20 14:59:25,982 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [1d302cac] Correlation ID: 1d302cac, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-10-20 14:59:26,013 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) [6e44a481] Correlation ID: 6e44a481, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-10-20 14:59:26,020 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) [414f8566] Correlation ID: 414f8566, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-10-20 14:59:26,027 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) [783e0dc7] Correlation ID: 783e0dc7, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-10-20 14:59:26,034 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [27c867b8] Correlation ID: 27c867b8, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-10-20 14:59:26,083 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-8) [58087a79] Running command: LogoutUserCommand internal: false. >2014-10-20 14:59:26,092 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-8) [58087a79] Correlation ID: 58087a79, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-10-20 14:59:26,102 INFO [org.ovirt.engine.ui.frontend.server.gwt.plugin.PluginDataManager] (ajp-/127.0.0.1:8702-10) Reading UI plugin descriptor [/usr/share/ovirt-engine/ui-plugins/redhat_support_plugin_rhev.json] >2014-10-20 14:59:26,127 INFO [org.ovirt.engine.ui.frontend.server.gwt.plugin.PluginDataManager] (ajp-/127.0.0.1:8702-10) Reading UI plugin configuration [/etc/ovirt-engine/ui-plugins/redhat_support_plugin_rhev-config.json] >2014-10-20 14:59:26,193 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-10) Invalid locale found in configuration: >2014-10-20 14:59:26,193 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-10) Invalid locale found in configuration: >2014-10-20 14:59:27,168 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-9) Initializing Host: rose05.qa.lab.tlv.redhat.com >2014-10-20 14:59:27,273 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-10) Initializing Host: master-vds10.qa.lab.tlv.redhat.com >2014-10-20 14:59:27,367 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-20 14:59:27,925 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-20 14:59:27,926 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to /10.35.70.4 >2014-10-20 14:59:28,960 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-8) START, GetHardwareInfoVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]), log id: 7de0b503 >2014-10-20 14:59:28,981 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-8) FINISH, GetHardwareInfoVDSCommand, log id: 7de0b503 >2014-10-20 14:59:29,216 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 14:59:29,242 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 14:59:29,336 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-8) [7b0fee1d] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: d5b2492d-8110-4614-9ce4-4a7a118c7942 Type: VDS >2014-10-20 14:59:29,345 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-8) [7b0fee1d] START, SetVdsStatusVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, status=NonOperational, nonOperationalReason=EMULATED_MACHINES_INCOMPATIBLE_WITH_CLUSTER, stopSpmFailureLogged=false), log id: 7ed55bb9 >2014-10-20 14:59:29,447 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-8) [7b0fee1d] FINISH, SetVdsStatusVDSCommand, log id: 7ed55bb9 >2014-10-20 14:59:29,462 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-11) START, GetHardwareInfoVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, vds=Host[cyan-vdsg.qa.lab.tlv.redhat.com,3a2a45bd-4411-4428-aa45-a514d30d3ea6]), log id: f4c37bc >2014-10-20 14:59:29,482 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-11) FINISH, GetHardwareInfoVDSCommand, log id: f4c37bc >2014-10-20 14:59:29,492 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-7) START, GetHardwareInfoVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c]), log id: 363c9b4b >2014-10-20 14:59:29,507 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-7) FINISH, GetHardwareInfoVDSCommand, log id: 363c9b4b >2014-10-20 14:59:29,715 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-8) [7b0fee1d] Correlation ID: 7b0fee1d, Job ID: 27dc1cf4-5a83-4488-9392-9c4f445b0ecd, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com does not comply with the cluster cl_35_ppc emulated machines. The Hosts emulated machines are rhel6.5.0 and the cluster is [rhel6.5.0, pseries]} >2014-10-20 14:59:29,867 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-8) [7b0fee1d] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host alma07.qa.lab.tlv.redhat.com was set to NonOperational. >2014-10-20 14:59:30,045 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-11) Initializing Host: cyan-vdsg.qa.lab.tlv.redhat.com >2014-10-20 14:59:30,087 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-7) Initializing Host: alma06.qa.lab.tlv.redhat.com >2014-10-20 14:59:30,106 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-8) [7b0fee1d] Initializing Host: alma07.qa.lab.tlv.redhat.com >2014-10-20 14:59:30,115 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-8) [7de40ed8] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: d5b2492d-8110-4614-9ce4-4a7a118c7942 Type: VDS >2014-10-20 14:59:30,158 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-11) [6b6a3346] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 3a2a45bd-4411-4428-aa45-a514d30d3ea6 Type: VDS >2014-10-20 14:59:30,172 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-8) [7de40ed8] Host d5b2492d-8110-4614-9ce4-4a7a118c7942 : alma07.qa.lab.tlv.redhat.com is already in NonOperational status for reason EMULATED_MACHINES_INCOMPATIBLE_WITH_CLUSTER. SetNonOperationalVds command is skipped. >2014-10-20 14:59:30,221 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-7) [4ddf2d33] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 1c39bd88-8672-408f-a593-60d6ef8f647c Type: VDS >2014-10-20 14:59:34,149 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-31) [58a4948] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: f8f5eaee-8fd0-4b45-87db-62d61b03a916 Type: StoragePool >2014-10-20 14:59:34,158 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-31) [58a4948] Storage Pool f8f5eaee-8fd0-4b45-87db-62d61b03a916 - Updating Storage Domain a1341057-9cf8-4750-a734-656d05a5d2c5 status from Active to Unknown, reason : null >2014-10-20 14:59:34,188 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-31) [58a4948] Correlation ID: 58a4948, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center dc_35. Setting status to Non Responsive. >2014-10-20 14:59:34,382 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-31) [58a4948] hostFromVds::selectedVds - alma06.qa.lab.tlv.redhat.com, spmStatus Free, storage pool dc_35 >2014-10-20 14:59:34,395 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-31) [58a4948] Found spm host 3a2a45bd-4411-4428-aa45-a514d30d3ea6, host name: cyan-vdsg.qa.lab.tlv.redhat.com, according to spmId: 5. >2014-10-20 14:59:34,421 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-31) [58a4948] SpmStatus on vds 3a2a45bd-4411-4428-aa45-a514d30d3ea6: SPM >2014-10-20 14:59:34,421 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-31) [58a4948] Using old spm server: cyan-vdsg.qa.lab.tlv.redhat.com, no start needed >2014-10-20 14:59:34,461 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-31) [58a4948] Initialize Irs proxy from vds: 10.35.70.4 >2014-10-20 14:59:34,478 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-31) [58a4948] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Storage Pool Manager runs on Host cyan-vdsg.qa.lab.tlv.redhat.com (Address: 10.35.70.4). >2014-10-20 14:59:34,483 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to /10.35.70.4 >2014-10-20 14:59:34,679 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-13) [58a4948] START, SPMGetAllTasksInfoVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false), log id: 6d11e996 >2014-10-20 14:59:34,891 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-13) [58a4948] -- executeIrsBrokerCommand: Attempting on storage pool f8f5eaee-8fd0-4b45-87db-62d61b03a916 >2014-10-20 14:59:34,903 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-13) [58a4948] START, HSMGetAllTasksInfoVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6), log id: 3d373b59 >2014-10-20 14:59:34,914 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-13) [58a4948] FINISH, HSMGetAllTasksInfoVDSCommand, return: [], log id: 3d373b59 >2014-10-20 14:59:34,914 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-13) [58a4948] FINISH, SPMGetAllTasksInfoVDSCommand, return: [], log id: 6d11e996 >2014-10-20 14:59:34,914 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-7-thread-13) [58a4948] Discovered no tasks on Storage Pool dc_35 >2014-10-20 15:02:05,168 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/usr/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf". >2014-10-20 15:02:05,169 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) The file "/etc/ovirt-engine/engine.conf" doesn't exist or isn't readable. Will return an empty set of properties. >2014-10-20 15:02:05,169 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/10-rhevm-doc.conf". >2014-10-20 15:02:05,170 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-database.conf". >2014-10-20 15:02:05,170 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf". >2014-10-20 15:02:05,171 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf". >2014-10-20 15:02:05,171 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf". >2014-10-20 15:02:05,171 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/20-branding-rhev.conf". >2014-10-20 15:02:05,172 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/50-redhat-support-plugin.conf". >2014-10-20 15:02:05,173 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_AJP_ENABLED" is "true". >2014-10-20 15:02:05,173 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_AJP_PORT" is "8702". >2014-10-20 15:02:05,173 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_APPS" is "engine.ear restapi.war legacy_restapi.war "/usr/share/ovirt-engine/branding/rhev-2.brand/applications/rhev.ear" redhat-support-plugin-rhev.ear". >2014-10-20 15:02:05,173 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_CACHE" is "/var/cache/ovirt-engine". >2014-10-20 15:02:05,198 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_CHECK_INTERVAL" is "1000". >2014-10-20 15:02:05,198 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_CONNECTION_TIMEOUT" is "300000". >2014-10-20 15:02:05,198 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_DATABASE" is "engine". >2014-10-20 15:02:05,199 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_DRIVER" is "org.postgresql.Driver". >2014-10-20 15:02:05,199 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_HOST" is "localhost". >2014-10-20 15:02:05,199 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_MAX_CONNECTIONS" is "100". >2014-10-20 15:02:05,199 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_MIN_CONNECTIONS" is "1". >2014-10-20 15:02:05,200 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_PASSWORD" is "***". >2014-10-20 15:02:05,200 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_PORT" is "5432". >2014-10-20 15:02:05,200 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_SECURED" is "False". >2014-10-20 15:02:05,200 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_SECURED_VALIDATION" is "False". >2014-10-20 15:02:05,200 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_URL" is "jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory". >2014-10-20 15:02:05,201 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_USER" is "engine". >2014-10-20 15:02:05,201 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DEBUG_ADDRESS" is "". >2014-10-20 15:02:05,201 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DOC" is "/usr/share/doc/ovirt-engine". >2014-10-20 15:02:05,201 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_ETC" is "/etc/ovirt-engine". >2014-10-20 15:02:05,202 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_EXTENSION_PATH" is "/usr/share/ovirt-engine/extensions.d:/etc/ovirt-engine/extensions.d". >2014-10-20 15:02:05,202 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_FQDN" is "dhcp163-98.scl.lab.tlv.redhat.com". >2014-10-20 15:02:05,202 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_GROUP" is "ovirt". >2014-10-20 15:02:05,202 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HEAP_MAX" is "1g". >2014-10-20 15:02:05,202 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HEAP_MIN" is "1g". >2014-10-20 15:02:05,203 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HTTPS_ENABLED" is "false". >2014-10-20 15:02:05,203 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HTTPS_PORT" is "None". >2014-10-20 15:02:05,203 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HTTPS_PROTOCOLS" is "SSLv3,TLSv1,TLSv1.1,TLSv1.2". >2014-10-20 15:02:05,203 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HTTP_ENABLED" is "false". >2014-10-20 15:02:05,204 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HTTP_PORT" is "None". >2014-10-20 15:02:05,204 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_JAVA_MODULEPATH" is "/usr/share/ovirt-engine/modules". >2014-10-20 15:02:05,204 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_JVM_ARGS" is " -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/var/log/ovirt-engine/dump"". >2014-10-20 15:02:05,204 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_LOG" is "/var/log/ovirt-engine". >2014-10-20 15:02:05,204 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_LOG_TO_CONSOLE" is "false". >2014-10-20 15:02:05,205 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_MANUAL" is "/usr/share/doc/rhevm-doc/manual". >2014-10-20 15:02:05,205 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PERM_MAX" is "256m". >2014-10-20 15:02:05,205 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PERM_MIN" is "256m". >2014-10-20 15:02:05,205 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI" is "/etc/pki/ovirt-engine". >2014-10-20 15:02:05,206 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_CA" is "/etc/pki/ovirt-engine/ca.pem". >2014-10-20 15:02:05,206 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_ENGINE_CERT" is "/etc/pki/ovirt-engine/certs/engine.cer". >2014-10-20 15:02:05,206 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_ENGINE_STORE" is "/etc/pki/ovirt-engine/keys/engine.p12". >2014-10-20 15:02:05,206 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_ENGINE_STORE_ALIAS" is "1". >2014-10-20 15:02:05,207 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_ENGINE_STORE_PASSWORD" is "***". >2014-10-20 15:02:05,207 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_TRUST_STORE" is "/etc/pki/ovirt-engine/.truststore". >2014-10-20 15:02:05,207 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_TRUST_STORE_PASSWORD" is "***". >2014-10-20 15:02:05,207 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PROPERTIES" is " jsse.enableSNIExtension=false "java.security.krb5.conf=/etc/ovirt-engine/krb5.conf"". >2014-10-20 15:02:05,207 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PROXY_ENABLED" is "true". >2014-10-20 15:02:05,208 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PROXY_HTTPS_PORT" is "443". >2014-10-20 15:02:05,208 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PROXY_HTTP_PORT" is "80". >2014-10-20 15:02:05,208 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_REPORTS_BASE_URL" is "". >2014-10-20 15:02:05,208 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_REPORTS_DASHBOARD_URL" is "/flow.html?viewAsDashboardFrame=true". >2014-10-20 15:02:05,231 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_REPORTS_NOT_INSTALLED_URL" is "/ovirt-engine/ReportsNotInstalled.html". >2014-10-20 15:02:05,231 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_REPORTS_PROXY_URL" is "/ovirt/reports-interface". >2014-10-20 15:02:05,231 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_REPORTS_READ_TIMEOUT" is "". >2014-10-20 15:02:05,232 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_REPORTS_RIGHTCLICK_URL" is "". >2014-10-20 15:02:05,232 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_REPORTS_VERIFY_CHAIN" is "true". >2014-10-20 15:02:05,232 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_REPORTS_VERIFY_HOST" is "true". >2014-10-20 15:02:05,232 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_STOP_INTERVAL" is "1". >2014-10-20 15:02:05,232 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_STOP_TIME" is "10". >2014-10-20 15:02:05,233 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_TMP" is "/var/tmp/ovirt-engine". >2014-10-20 15:02:05,233 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_UP_MARK" is "/var/lib/ovirt-engine/engine.up". >2014-10-20 15:02:05,233 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_URI" is "/ovirt-engine". >2014-10-20 15:02:05,233 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_USER" is "ovirt". >2014-10-20 15:02:05,233 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_USR" is "/usr/share/ovirt-engine". >2014-10-20 15:02:05,234 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_VAR" is "/var/lib/ovirt-engine". >2014-10-20 15:02:05,234 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_VERBOSE_GC" is "false". >2014-10-20 15:02:05,234 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "JBOSS_HOME" is "/usr/share/jbossas". >2014-10-20 15:02:05,234 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "JBOSS_RUNTIME" is "/var/lib/ovirt-engine/jboss_runtime". >2014-10-20 15:02:05,234 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "SENSITIVE_KEYS" is ",ENGINE_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD". >2014-10-20 15:02:18,187 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 50) Start initializing Backend >2014-10-20 15:02:19,077 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (ServerService Thread Pool -- 50) Loaded file /etc/ovirt-engine/osinfo.conf.d/00-defaults.properties >2014-10-20 15:02:19,545 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 50) Running ovirt-engine 3.5.0-0.14.beta.el6ev >2014-10-20 15:02:19,546 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ServerService Thread Pool -- 50) Start initializing dictionaries >2014-10-20 15:02:19,550 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ServerService Thread Pool -- 50) Finished initializing dictionaries >2014-10-20 15:02:19,550 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 50) Start initializing AuditLogCleanupManager >2014-10-20 15:02:19,551 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 50) Setting audit cleanup manager to run at: 35 35 3 * * ? >2014-10-20 15:02:19,594 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 50) Finished initializing AuditLogCleanupManager >2014-10-20 15:02:19,602 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 50) Start initializing CommandEntityCleanupManager >2014-10-20 15:02:19,603 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 50) Setting command entity cleanup manager to run at: 35 35 3 * * ? >2014-10-20 15:02:19,605 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 50) Finished initializing CommandEntityCleanupManager >2014-10-20 15:02:19,608 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 50) Start initializing TagsDirector >2014-10-20 15:02:19,632 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 50) Tag root added to tree >2014-10-20 15:02:19,659 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 50) Finished initializing TagsDirector >2014-10-20 15:02:19,660 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (ServerService Thread Pool -- 50) Start initializing IsoDomainListSyncronizer >2014-10-20 15:02:19,672 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (ServerService Thread Pool -- 50) Finished initializing IsoDomainListSyncronizer >2014-10-20 15:02:19,760 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 50) Completed initializing handlers >2014-10-20 15:02:19,832 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 50) Start initializing ErrorTranslatorImpl >2014-10-20 15:02:19,883 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 50) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table. >2014-10-20 15:02:19,884 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 50) Finished initializing ErrorTranslatorImpl >2014-10-20 15:02:19,884 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 50) Start initializing ErrorTranslatorImpl >2014-10-20 15:02:19,885 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 50) Finished initializing ErrorTranslatorImpl >2014-10-20 15:02:19,885 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 50) Mark incomplete jobs as UNKNOWN >2014-10-20 15:02:20,040 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ServerService Thread Pool -- 50) Start initializing JobRepositoryCleanupManager >2014-10-20 15:02:20,041 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ServerService Thread Pool -- 50) Finished initializing JobRepositoryCleanupManager >2014-10-20 15:02:20,042 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ServerService Thread Pool -- 50) Start initializing AutoRecoveryManager >2014-10-20 15:02:20,043 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ServerService Thread Pool -- 50) Finished initializing AutoRecoveryManager >2014-10-20 15:02:20,046 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ServerService Thread Pool -- 50) Start initializing ExecutionMessageDirector >2014-10-20 15:02:20,048 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ServerService Thread Pool -- 50) Finished initializing ExecutionMessageDirector >2014-10-20 15:02:20,415 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 50) Loading extension 'builtin-authn-internal' >2014-10-20 15:02:20,417 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 50) Extension 'builtin-authn-internal' loaded >2014-10-20 15:02:20,446 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 50) Loading extension 'internal' >2014-10-20 15:02:20,453 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 50) Extension 'internal' loaded >2014-10-20 15:02:20,460 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 50) Loading extension 'builtin-authn-qa.lab.tlv.redhat.com' >2014-10-20 15:02:20,478 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 50) Extension 'builtin-authn-qa.lab.tlv.redhat.com' loaded >2014-10-20 15:02:20,480 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 50) Loading extension 'qa.lab.tlv.redhat.com' >2014-10-20 15:02:20,481 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 50) Extension 'qa.lab.tlv.redhat.com' loaded >2014-10-20 15:02:20,482 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 50) Initializing extension 'qa.lab.tlv.redhat.com' >2014-10-20 15:02:20,482 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 50) Extension 'qa.lab.tlv.redhat.com' initialized >2014-10-20 15:02:20,487 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 50) Initializing extension 'builtin-authn-internal' >2014-10-20 15:02:20,487 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 50) Extension 'builtin-authn-internal' initialized >2014-10-20 15:02:20,488 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 50) Initializing extension 'builtin-authn-qa.lab.tlv.redhat.com' >2014-10-20 15:02:20,488 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 50) Extension 'builtin-authn-qa.lab.tlv.redhat.com' initialized >2014-10-20 15:02:20,489 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 50) Initializing extension 'internal' >2014-10-20 15:02:20,489 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 50) Extension 'internal' initialized >2014-10-20 15:02:20,489 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 50) Start of enabled extensions list >2014-10-20 15:02:20,489 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 50) Instance name: 'qa.lab.tlv.redhat.com', Extension name: 'Kerberos/Ldap Authz (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-10-20 15:02:20,489 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 50) Instance name: 'builtin-authn-internal', Extension name: 'Internal Authn (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-10-20 15:02:20,490 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 50) Instance name: 'builtin-authn-qa.lab.tlv.redhat.com', Extension name: 'Kerberos/Ldap Authn (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-10-20 15:02:20,490 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 50) Instance name: 'internal', Extension name: 'Internal Authz (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-10-20 15:02:20,490 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 50) End of enabled extensions list >2014-10-20 15:02:20,770 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (ServerService Thread Pool -- 50) Start initializing DbUserCacheManager >2014-10-20 15:02:20,771 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (ServerService Thread Pool -- 50) Finished initializing DbUserCacheManager >2014-10-20 15:02:20,789 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (ServerService Thread Pool -- 50) Initialization of AsyncTaskManager completed successfully. >2014-10-20 15:02:20,792 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 50) Start initializing ResourceManager >2014-10-20 15:02:21,049 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 50) Entered VdsManager constructor >2014-10-20 15:02:21,091 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 50) Initialize vdsBroker (alma06.qa.lab.tlv.redhat.com,54,321) >2014-10-20 15:02:21,326 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 50) VDS 1c39bd88-8672-408f-a593-60d6ef8f647c was added to the Resource Manager >2014-10-20 15:02:21,327 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 50) Entered VdsManager constructor >2014-10-20 15:02:21,345 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 50) Initialize vdsBroker (alma07.qa.lab.tlv.redhat.com,54,321) >2014-10-20 15:02:21,365 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 50) VDS d5b2492d-8110-4614-9ce4-4a7a118c7942 was added to the Resource Manager >2014-10-20 15:02:21,366 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 50) Entered VdsManager constructor >2014-10-20 15:02:21,402 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 50) Initialize vdsBroker (10.35.70.4,54,321) >2014-10-20 15:02:21,420 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 50) VDS 3a2a45bd-4411-4428-aa45-a514d30d3ea6 was added to the Resource Manager >2014-10-20 15:02:21,420 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 50) Entered VdsManager constructor >2014-10-20 15:02:21,450 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 50) Initialize vdsBroker (rose05.qa.lab.tlv.redhat.com,54,321) >2014-10-20 15:02:21,483 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 50) VDS 45869c56-4fbd-466b-977a-81604a392b78 was added to the Resource Manager >2014-10-20 15:02:21,483 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 50) Entered VdsManager constructor >2014-10-20 15:02:21,522 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 50) Initialize vdsBroker (master-vds10.qa.lab.tlv.redhat.com,54,321) >2014-10-20 15:02:21,541 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 50) VDS 51fdbad1-2bd5-47d4-aa62-4054e71fbbbf was added to the Resource Manager >2014-10-20 15:02:21,618 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 50) Finished initializing ResourceManager >2014-10-20 15:02:21,626 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (ServerService Thread Pool -- 50) Initialization of OvfDataUpdater completed successfully. >2014-10-20 15:02:21,626 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 50) Start scheduling to enable vds load balancer >2014-10-20 15:02:21,627 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 50) Finished scheduling to enable vds load balancer >2014-10-20 15:02:21,627 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 50) Start HA Reservation check >2014-10-20 15:02:21,640 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 50) Finished HA Reservation check >2014-10-20 15:02:21,669 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 50) Init VM custom properties utilities >2014-10-20 15:02:21,670 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 50) Init device custom properties utilities >2014-10-20 15:02:21,675 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 50) Initializing Scheduling manager >2014-10-20 15:02:21,685 INFO [org.ovirt.engine.core.bll.network.MacPoolManagerRanges] (org.ovirt.thread.pool-7-thread-1) Start initializing MacPoolManagerRanges >2014-10-20 15:02:21,729 INFO [org.ovirt.engine.core.bll.network.MacPoolManagerRanges] (org.ovirt.thread.pool-7-thread-1) Finished initializing. Available MACs in pool: 246 >2014-10-20 15:02:21,754 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 50) Starting external scheduler dicovery thread >2014-10-20 15:02:21,760 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 50) Initialized Scheduling manager >2014-10-20 15:02:21,760 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (ServerService Thread Pool -- 50) Initializing DWH Heart Beat >2014-10-20 15:02:21,761 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (ServerService Thread Pool -- 50) DWH Heart Beat initialized >2014-10-20 15:02:21,956 INFO [org.ovirt.engine.core.bll.scheduling.external.ExternalSchedulerDiscoveryThread] (Thread-69) PolicyUnits updated >2014-10-20 15:02:24,426 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-7) Initializing Host: alma07.qa.lab.tlv.redhat.com >2014-10-20 15:02:24,598 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to /10.35.70.4 >2014-10-20 15:02:24,621 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-11) Initializing Host: master-vds10.qa.lab.tlv.redhat.com >2014-10-20 15:02:24,630 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-20 15:02:24,668 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-10) Initializing Host: rose05.qa.lab.tlv.redhat.com >2014-10-20 15:02:25,314 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-8) AuditLogType: UNASSIGNED not exist in string table >2014-10-20 15:02:25,315 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-8) AuditLogType: VDS_HIGH_NETWORK_USE not exist in string table >2014-10-20 15:02:25,315 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-8) AuditLogType: USER_FAILED_REMOVE_VM not exist in string table >2014-10-20 15:02:25,316 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-8) AuditLogType: USER_RUN_UNLOCK_ENTITY_SCRIPT not exist in string table >2014-10-20 15:02:25,317 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-8) AuditLogType: VDS_NETWORK_MTU_DIFFER_FROM_LOGICAL_NETWORK not exist in string table >2014-10-20 15:02:25,317 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-8) AuditLogType: STORAGE_ACTIVATE_ASYNC not exist in string table >2014-10-20 15:02:25,318 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-8) AuditLogType: USER_ADDED_DISK_PROFILE not exist in string table >2014-10-20 15:02:25,318 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-8) AuditLogType: USER_FAILED_TO_ADD_DISK_PROFILE not exist in string table >2014-10-20 15:02:25,318 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-8) AuditLogType: USER_REMOVED_DISK_PROFILE not exist in string table >2014-10-20 15:02:25,318 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-8) AuditLogType: USER_FAILED_TO_REMOVE_DISK_PROFILE not exist in string table >2014-10-20 15:02:25,318 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-8) AuditLogType: USER_UPDATED_DISK_PROFILE not exist in string table >2014-10-20 15:02:25,318 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-8) AuditLogType: USER_FAILED_TO_UPDATE_DISK_PROFILE not exist in string table >2014-10-20 15:02:25,318 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-8) AuditLogType: USER_ADDED_CPU_PROFILE not exist in string table >2014-10-20 15:02:25,318 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-8) AuditLogType: USER_FAILED_TO_ADD_CPU_PROFILE not exist in string table >2014-10-20 15:02:25,319 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-8) AuditLogType: USER_REMOVED_CPU_PROFILE not exist in string table >2014-10-20 15:02:25,319 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-8) AuditLogType: USER_FAILED_TO_REMOVE_CPU_PROFILE not exist in string table >2014-10-20 15:02:25,319 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-8) AuditLogType: USER_UPDATED_CPU_PROFILE not exist in string table >2014-10-20 15:02:25,319 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-8) AuditLogType: USER_FAILED_TO_UPDATE_CPU_PROFILE not exist in string table >2014-10-20 15:02:25,704 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-8) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 15:02:25,732 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 15:02:25,946 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-6) START, GetHardwareInfoVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c]), log id: 899692a >2014-10-20 15:02:25,965 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-6) FINISH, GetHardwareInfoVDSCommand, log id: 899692a >2014-10-20 15:02:26,030 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-8) START, GetHardwareInfoVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, vds=Host[cyan-vdsg.qa.lab.tlv.redhat.com,3a2a45bd-4411-4428-aa45-a514d30d3ea6]), log id: 571aff79 >2014-10-20 15:02:26,064 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-8) FINISH, GetHardwareInfoVDSCommand, log id: 571aff79 >2014-10-20 15:02:26,594 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Initializing Host: alma06.qa.lab.tlv.redhat.com >2014-10-20 15:02:26,599 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-8) Initializing Host: cyan-vdsg.qa.lab.tlv.redhat.com >2014-10-20 15:02:26,881 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-6) [6e77dd17] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 1c39bd88-8672-408f-a593-60d6ef8f647c Type: VDS >2014-10-20 15:02:26,882 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-8) [2f54e4c7] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 3a2a45bd-4411-4428-aa45-a514d30d3ea6 Type: VDS >2014-10-20 15:02:31,686 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-32) [1444510c] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: f8f5eaee-8fd0-4b45-87db-62d61b03a916 Type: StoragePool >2014-10-20 15:02:31,695 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-32) [1444510c] Storage Pool f8f5eaee-8fd0-4b45-87db-62d61b03a916 - Updating Storage Domain a1341057-9cf8-4750-a734-656d05a5d2c5 status from Active to Unknown, reason : null >2014-10-20 15:02:31,774 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-32) [1444510c] Correlation ID: 1444510c, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center dc_35. Setting status to Non Responsive. >2014-10-20 15:02:31,936 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-32) [1444510c] hostFromVds::selectedVds - alma06.qa.lab.tlv.redhat.com, spmStatus Free, storage pool dc_35 >2014-10-20 15:02:31,946 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-32) [1444510c] Found spm host 3a2a45bd-4411-4428-aa45-a514d30d3ea6, host name: cyan-vdsg.qa.lab.tlv.redhat.com, according to spmId: 5. >2014-10-20 15:02:31,973 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-32) [1444510c] SpmStatus on vds 3a2a45bd-4411-4428-aa45-a514d30d3ea6: SPM >2014-10-20 15:02:31,973 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-32) [1444510c] Using old spm server: cyan-vdsg.qa.lab.tlv.redhat.com, no start needed >2014-10-20 15:02:31,982 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-32) [1444510c] Initialize Irs proxy from vds: 10.35.70.4 >2014-10-20 15:02:31,993 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-32) [1444510c] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Storage Pool Manager runs on Host cyan-vdsg.qa.lab.tlv.redhat.com (Address: 10.35.70.4). >2014-10-20 15:02:32,002 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to /10.35.70.4 >2014-10-20 15:02:32,013 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-10) [1444510c] START, SPMGetAllTasksInfoVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false), log id: 3082db1e >2014-10-20 15:02:32,255 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-10) [1444510c] -- executeIrsBrokerCommand: Attempting on storage pool f8f5eaee-8fd0-4b45-87db-62d61b03a916 >2014-10-20 15:02:32,266 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-10) [1444510c] START, HSMGetAllTasksInfoVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6), log id: 5939229c >2014-10-20 15:02:32,280 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-10) [1444510c] FINISH, HSMGetAllTasksInfoVDSCommand, return: [], log id: 5939229c >2014-10-20 15:02:32,281 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-10) [1444510c] FINISH, SPMGetAllTasksInfoVDSCommand, return: [], log id: 3082db1e >2014-10-20 15:02:32,281 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-7-thread-10) [1444510c] Discovered no tasks on Storage Pool dc_35 >2014-10-20 15:03:52,319 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Loaded file "/usr/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf". >2014-10-20 15:03:52,320 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) The file "/etc/ovirt-engine/engine.conf" doesn't exist or isn't readable. Will return an empty set of properties. >2014-10-20 15:03:52,320 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Loaded file "/etc/ovirt-engine/engine.conf.d/10-rhevm-doc.conf". >2014-10-20 15:03:52,321 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-database.conf". >2014-10-20 15:03:52,321 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf". >2014-10-20 15:03:52,322 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf". >2014-10-20 15:03:52,322 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf". >2014-10-20 15:03:52,322 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Loaded file "/etc/ovirt-engine/engine.conf.d/20-branding-rhev.conf". >2014-10-20 15:03:52,323 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Loaded file "/etc/ovirt-engine/engine.conf.d/50-redhat-support-plugin.conf". >2014-10-20 15:03:52,324 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_AJP_ENABLED" is "true". >2014-10-20 15:03:52,324 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_AJP_PORT" is "8702". >2014-10-20 15:03:52,324 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_APPS" is "engine.ear restapi.war legacy_restapi.war "/usr/share/ovirt-engine/branding/rhev-2.brand/applications/rhev.ear" redhat-support-plugin-rhev.ear". >2014-10-20 15:03:52,324 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_CACHE" is "/var/cache/ovirt-engine". >2014-10-20 15:03:52,324 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_DB_CHECK_INTERVAL" is "1000". >2014-10-20 15:03:52,325 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_DB_CONNECTION_TIMEOUT" is "300000". >2014-10-20 15:03:52,325 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_DB_DATABASE" is "engine". >2014-10-20 15:03:52,325 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_DB_DRIVER" is "org.postgresql.Driver". >2014-10-20 15:03:52,325 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_DB_HOST" is "localhost". >2014-10-20 15:03:52,325 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_DB_MAX_CONNECTIONS" is "100". >2014-10-20 15:03:52,326 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_DB_MIN_CONNECTIONS" is "1". >2014-10-20 15:03:52,326 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_DB_PASSWORD" is "***". >2014-10-20 15:03:52,326 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_DB_PORT" is "5432". >2014-10-20 15:03:52,326 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_DB_SECURED" is "False". >2014-10-20 15:03:52,345 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_DB_SECURED_VALIDATION" is "False". >2014-10-20 15:03:52,346 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_DB_URL" is "jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory". >2014-10-20 15:03:52,346 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_DB_USER" is "engine". >2014-10-20 15:03:52,346 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_DEBUG_ADDRESS" is "". >2014-10-20 15:03:52,346 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_DOC" is "/usr/share/doc/ovirt-engine". >2014-10-20 15:03:52,346 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_ETC" is "/etc/ovirt-engine". >2014-10-20 15:03:52,347 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_EXTENSION_PATH" is "/usr/share/ovirt-engine/extensions.d:/etc/ovirt-engine/extensions.d". >2014-10-20 15:03:52,347 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_FQDN" is "dhcp163-98.scl.lab.tlv.redhat.com". >2014-10-20 15:03:52,347 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_GROUP" is "ovirt". >2014-10-20 15:03:52,347 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_HEAP_MAX" is "1g". >2014-10-20 15:03:52,347 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_HEAP_MIN" is "1g". >2014-10-20 15:03:52,348 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_HTTPS_ENABLED" is "false". >2014-10-20 15:03:52,348 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_HTTPS_PORT" is "None". >2014-10-20 15:03:52,348 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_HTTPS_PROTOCOLS" is "SSLv3,TLSv1,TLSv1.1,TLSv1.2". >2014-10-20 15:03:52,348 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_HTTP_ENABLED" is "false". >2014-10-20 15:03:52,348 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_HTTP_PORT" is "None". >2014-10-20 15:03:52,349 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_JAVA_MODULEPATH" is "/usr/share/ovirt-engine/modules". >2014-10-20 15:03:52,349 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_JVM_ARGS" is " -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/var/log/ovirt-engine/dump"". >2014-10-20 15:03:52,349 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_LOG" is "/var/log/ovirt-engine". >2014-10-20 15:03:52,349 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_LOG_TO_CONSOLE" is "false". >2014-10-20 15:03:52,349 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_MANUAL" is "/usr/share/doc/rhevm-doc/manual". >2014-10-20 15:03:52,350 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_PERM_MAX" is "256m". >2014-10-20 15:03:52,350 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_PERM_MIN" is "256m". >2014-10-20 15:03:52,350 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_PKI" is "/etc/pki/ovirt-engine". >2014-10-20 15:03:52,350 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_PKI_CA" is "/etc/pki/ovirt-engine/ca.pem". >2014-10-20 15:03:52,351 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_PKI_ENGINE_CERT" is "/etc/pki/ovirt-engine/certs/engine.cer". >2014-10-20 15:03:52,351 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_PKI_ENGINE_STORE" is "/etc/pki/ovirt-engine/keys/engine.p12". >2014-10-20 15:03:52,351 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_PKI_ENGINE_STORE_ALIAS" is "1". >2014-10-20 15:03:52,351 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_PKI_ENGINE_STORE_PASSWORD" is "***". >2014-10-20 15:03:52,351 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_PKI_TRUST_STORE" is "/etc/pki/ovirt-engine/.truststore". >2014-10-20 15:03:52,352 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_PKI_TRUST_STORE_PASSWORD" is "***". >2014-10-20 15:03:52,352 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_PROPERTIES" is " jsse.enableSNIExtension=false "java.security.krb5.conf=/etc/ovirt-engine/krb5.conf"". >2014-10-20 15:03:52,352 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_PROXY_ENABLED" is "true". >2014-10-20 15:03:52,352 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_PROXY_HTTPS_PORT" is "443". >2014-10-20 15:03:52,352 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_PROXY_HTTP_PORT" is "80". >2014-10-20 15:03:52,353 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_REPORTS_BASE_URL" is "". >2014-10-20 15:03:52,353 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_REPORTS_DASHBOARD_URL" is "/flow.html?viewAsDashboardFrame=true". >2014-10-20 15:03:52,353 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_REPORTS_NOT_INSTALLED_URL" is "/ovirt-engine/ReportsNotInstalled.html". >2014-10-20 15:03:52,353 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_REPORTS_PROXY_URL" is "/ovirt/reports-interface". >2014-10-20 15:03:52,353 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_REPORTS_READ_TIMEOUT" is "". >2014-10-20 15:03:52,354 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_REPORTS_RIGHTCLICK_URL" is "". >2014-10-20 15:03:52,354 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_REPORTS_VERIFY_CHAIN" is "true". >2014-10-20 15:03:52,354 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_REPORTS_VERIFY_HOST" is "true". >2014-10-20 15:03:52,354 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_STOP_INTERVAL" is "1". >2014-10-20 15:03:52,354 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_STOP_TIME" is "10". >2014-10-20 15:03:52,355 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_TMP" is "/var/tmp/ovirt-engine". >2014-10-20 15:03:52,355 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_UP_MARK" is "/var/lib/ovirt-engine/engine.up". >2014-10-20 15:03:52,355 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_URI" is "/ovirt-engine". >2014-10-20 15:03:52,355 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_USER" is "ovirt". >2014-10-20 15:03:52,383 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_USR" is "/usr/share/ovirt-engine". >2014-10-20 15:03:52,383 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_VAR" is "/var/lib/ovirt-engine". >2014-10-20 15:03:52,383 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "ENGINE_VERBOSE_GC" is "false". >2014-10-20 15:03:52,384 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "JBOSS_HOME" is "/usr/share/jbossas". >2014-10-20 15:03:52,411 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "JBOSS_RUNTIME" is "/var/lib/ovirt-engine/jboss_runtime". >2014-10-20 15:03:52,411 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 41) Value of property "SENSITIVE_KEYS" is ",ENGINE_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD". >2014-10-20 15:04:04,295 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 52) Start initializing Backend >2014-10-20 15:04:05,359 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (ServerService Thread Pool -- 52) Loaded file /etc/ovirt-engine/osinfo.conf.d/00-defaults.properties >2014-10-20 15:04:06,093 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 52) Running ovirt-engine 3.5.0-0.14.beta.el6ev >2014-10-20 15:04:06,093 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ServerService Thread Pool -- 52) Start initializing dictionaries >2014-10-20 15:04:06,097 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ServerService Thread Pool -- 52) Finished initializing dictionaries >2014-10-20 15:04:06,097 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 52) Start initializing AuditLogCleanupManager >2014-10-20 15:04:06,098 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 52) Setting audit cleanup manager to run at: 35 35 3 * * ? >2014-10-20 15:04:06,155 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 52) Finished initializing AuditLogCleanupManager >2014-10-20 15:04:06,155 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 52) Start initializing CommandEntityCleanupManager >2014-10-20 15:04:06,156 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 52) Setting command entity cleanup manager to run at: 35 35 3 * * ? >2014-10-20 15:04:06,158 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 52) Finished initializing CommandEntityCleanupManager >2014-10-20 15:04:06,160 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 52) Start initializing TagsDirector >2014-10-20 15:04:06,222 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 52) Tag root added to tree >2014-10-20 15:04:06,268 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 52) Finished initializing TagsDirector >2014-10-20 15:04:06,269 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (ServerService Thread Pool -- 52) Start initializing IsoDomainListSyncronizer >2014-10-20 15:04:06,277 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (ServerService Thread Pool -- 52) Finished initializing IsoDomainListSyncronizer >2014-10-20 15:04:06,391 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 52) Completed initializing handlers >2014-10-20 15:04:06,505 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 52) Start initializing ErrorTranslatorImpl >2014-10-20 15:04:06,548 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 52) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table. >2014-10-20 15:04:06,550 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 52) Finished initializing ErrorTranslatorImpl >2014-10-20 15:04:06,550 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 52) Start initializing ErrorTranslatorImpl >2014-10-20 15:04:06,551 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 52) Finished initializing ErrorTranslatorImpl >2014-10-20 15:04:06,551 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 52) Mark incomplete jobs as UNKNOWN >2014-10-20 15:04:06,652 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ServerService Thread Pool -- 52) Start initializing JobRepositoryCleanupManager >2014-10-20 15:04:06,653 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ServerService Thread Pool -- 52) Finished initializing JobRepositoryCleanupManager >2014-10-20 15:04:06,653 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ServerService Thread Pool -- 52) Start initializing AutoRecoveryManager >2014-10-20 15:04:06,655 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ServerService Thread Pool -- 52) Finished initializing AutoRecoveryManager >2014-10-20 15:04:06,668 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ServerService Thread Pool -- 52) Start initializing ExecutionMessageDirector >2014-10-20 15:04:06,670 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ServerService Thread Pool -- 52) Finished initializing ExecutionMessageDirector >2014-10-20 15:04:06,833 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 52) Loading extension 'builtin-authn-internal' >2014-10-20 15:04:06,835 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 52) Extension 'builtin-authn-internal' loaded >2014-10-20 15:04:06,838 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 52) Loading extension 'internal' >2014-10-20 15:04:06,840 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 52) Extension 'internal' loaded >2014-10-20 15:04:06,847 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 52) Loading extension 'builtin-authn-qa.lab.tlv.redhat.com' >2014-10-20 15:04:06,866 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 52) Extension 'builtin-authn-qa.lab.tlv.redhat.com' loaded >2014-10-20 15:04:06,868 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 52) Loading extension 'qa.lab.tlv.redhat.com' >2014-10-20 15:04:06,880 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 52) Extension 'qa.lab.tlv.redhat.com' loaded >2014-10-20 15:04:06,881 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 52) Initializing extension 'qa.lab.tlv.redhat.com' >2014-10-20 15:04:06,882 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 52) Extension 'qa.lab.tlv.redhat.com' initialized >2014-10-20 15:04:06,883 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 52) Initializing extension 'builtin-authn-internal' >2014-10-20 15:04:06,883 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 52) Extension 'builtin-authn-internal' initialized >2014-10-20 15:04:06,884 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 52) Initializing extension 'builtin-authn-qa.lab.tlv.redhat.com' >2014-10-20 15:04:06,884 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 52) Extension 'builtin-authn-qa.lab.tlv.redhat.com' initialized >2014-10-20 15:04:06,884 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 52) Initializing extension 'internal' >2014-10-20 15:04:06,885 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 52) Extension 'internal' initialized >2014-10-20 15:04:06,885 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 52) Start of enabled extensions list >2014-10-20 15:04:06,885 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 52) Instance name: 'qa.lab.tlv.redhat.com', Extension name: 'Kerberos/Ldap Authz (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-10-20 15:04:06,885 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 52) Instance name: 'builtin-authn-internal', Extension name: 'Internal Authn (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-10-20 15:04:06,885 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 52) Instance name: 'builtin-authn-qa.lab.tlv.redhat.com', Extension name: 'Kerberos/Ldap Authn (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-10-20 15:04:06,886 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 52) Instance name: 'internal', Extension name: 'Internal Authz (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-10-20 15:04:06,886 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 52) End of enabled extensions list >2014-10-20 15:04:07,429 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (ServerService Thread Pool -- 52) Start initializing DbUserCacheManager >2014-10-20 15:04:07,430 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (ServerService Thread Pool -- 52) Finished initializing DbUserCacheManager >2014-10-20 15:04:07,483 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (ServerService Thread Pool -- 52) Initialization of AsyncTaskManager completed successfully. >2014-10-20 15:04:07,506 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 52) Start initializing ResourceManager >2014-10-20 15:04:07,875 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 52) Entered VdsManager constructor >2014-10-20 15:04:07,933 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 52) Initialize vdsBroker (alma07.qa.lab.tlv.redhat.com,54,321) >2014-10-20 15:04:08,220 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 52) VDS d5b2492d-8110-4614-9ce4-4a7a118c7942 was added to the Resource Manager >2014-10-20 15:04:08,220 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 52) Entered VdsManager constructor >2014-10-20 15:04:08,256 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 52) Initialize vdsBroker (10.35.70.4,54,321) >2014-10-20 15:04:08,270 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 52) VDS 3a2a45bd-4411-4428-aa45-a514d30d3ea6 was added to the Resource Manager >2014-10-20 15:04:08,271 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 52) Entered VdsManager constructor >2014-10-20 15:04:08,294 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 52) Initialize vdsBroker (rose05.qa.lab.tlv.redhat.com,54,321) >2014-10-20 15:04:08,317 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 52) VDS 45869c56-4fbd-466b-977a-81604a392b78 was added to the Resource Manager >2014-10-20 15:04:08,317 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 52) Entered VdsManager constructor >2014-10-20 15:04:08,343 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 52) Initialize vdsBroker (master-vds10.qa.lab.tlv.redhat.com,54,321) >2014-10-20 15:04:08,354 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 52) VDS 51fdbad1-2bd5-47d4-aa62-4054e71fbbbf was added to the Resource Manager >2014-10-20 15:04:08,355 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 52) Entered VdsManager constructor >2014-10-20 15:04:08,375 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 52) Initialize vdsBroker (alma06.qa.lab.tlv.redhat.com,54,321) >2014-10-20 15:04:08,393 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 52) VDS 1c39bd88-8672-408f-a593-60d6ef8f647c was added to the Resource Manager >2014-10-20 15:04:08,438 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 52) Finished initializing ResourceManager >2014-10-20 15:04:08,445 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (ServerService Thread Pool -- 52) Initialization of OvfDataUpdater completed successfully. >2014-10-20 15:04:08,446 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 52) Start scheduling to enable vds load balancer >2014-10-20 15:04:08,446 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 52) Finished scheduling to enable vds load balancer >2014-10-20 15:04:08,447 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 52) Start HA Reservation check >2014-10-20 15:04:08,448 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 52) Finished HA Reservation check >2014-10-20 15:04:08,462 INFO [org.ovirt.engine.core.bll.network.MacPoolManagerRanges] (org.ovirt.thread.pool-7-thread-1) Start initializing MacPoolManagerRanges >2014-10-20 15:04:08,478 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 52) Init VM custom properties utilities >2014-10-20 15:04:08,478 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 52) Init device custom properties utilities >2014-10-20 15:04:08,484 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 52) Initializing Scheduling manager >2014-10-20 15:04:08,548 INFO [org.ovirt.engine.core.bll.network.MacPoolManagerRanges] (org.ovirt.thread.pool-7-thread-1) Finished initializing. Available MACs in pool: 246 >2014-10-20 15:04:08,567 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 52) Starting external scheduler dicovery thread >2014-10-20 15:04:08,567 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 52) Initialized Scheduling manager >2014-10-20 15:04:08,568 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (ServerService Thread Pool -- 52) Initializing DWH Heart Beat >2014-10-20 15:04:08,569 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (ServerService Thread Pool -- 52) DWH Heart Beat initialized >2014-10-20 15:04:08,680 INFO [org.ovirt.engine.core.bll.scheduling.external.ExternalSchedulerDiscoveryThread] (Thread-70) PolicyUnits updated >2014-10-20 15:04:11,312 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-7) Initializing Host: alma07.qa.lab.tlv.redhat.com >2014-10-20 15:04:11,449 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-9) Initializing Host: rose05.qa.lab.tlv.redhat.com >2014-10-20 15:04:11,607 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-10) Initializing Host: master-vds10.qa.lab.tlv.redhat.com >2014-10-20 15:04:11,631 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to /10.35.70.4 >2014-10-20 15:04:11,644 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-20 15:04:12,416 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) AuditLogType: UNASSIGNED not exist in string table >2014-10-20 15:04:12,416 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) AuditLogType: VDS_HIGH_NETWORK_USE not exist in string table >2014-10-20 15:04:12,417 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) AuditLogType: USER_FAILED_REMOVE_VM not exist in string table >2014-10-20 15:04:12,417 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) AuditLogType: USER_RUN_UNLOCK_ENTITY_SCRIPT not exist in string table >2014-10-20 15:04:12,418 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) AuditLogType: VDS_NETWORK_MTU_DIFFER_FROM_LOGICAL_NETWORK not exist in string table >2014-10-20 15:04:12,419 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) AuditLogType: STORAGE_ACTIVATE_ASYNC not exist in string table >2014-10-20 15:04:12,419 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) AuditLogType: USER_ADDED_DISK_PROFILE not exist in string table >2014-10-20 15:04:12,419 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) AuditLogType: USER_FAILED_TO_ADD_DISK_PROFILE not exist in string table >2014-10-20 15:04:12,419 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) AuditLogType: USER_REMOVED_DISK_PROFILE not exist in string table >2014-10-20 15:04:12,419 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) AuditLogType: USER_FAILED_TO_REMOVE_DISK_PROFILE not exist in string table >2014-10-20 15:04:12,420 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) AuditLogType: USER_UPDATED_DISK_PROFILE not exist in string table >2014-10-20 15:04:12,420 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) AuditLogType: USER_FAILED_TO_UPDATE_DISK_PROFILE not exist in string table >2014-10-20 15:04:12,420 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) AuditLogType: USER_ADDED_CPU_PROFILE not exist in string table >2014-10-20 15:04:12,420 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) AuditLogType: USER_FAILED_TO_ADD_CPU_PROFILE not exist in string table >2014-10-20 15:04:12,420 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) AuditLogType: USER_REMOVED_CPU_PROFILE not exist in string table >2014-10-20 15:04:12,420 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) AuditLogType: USER_FAILED_TO_REMOVE_CPU_PROFILE not exist in string table >2014-10-20 15:04:12,421 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) AuditLogType: USER_UPDATED_CPU_PROFILE not exist in string table >2014-10-20 15:04:12,421 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) AuditLogType: USER_FAILED_TO_UPDATE_CPU_PROFILE not exist in string table >2014-10-20 15:04:13,042 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 15:04:13,053 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-8) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 15:04:13,282 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-8) START, GetHardwareInfoVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, vds=Host[cyan-vdsg.qa.lab.tlv.redhat.com,3a2a45bd-4411-4428-aa45-a514d30d3ea6]), log id: 2625a366 >2014-10-20 15:04:13,288 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-11) START, GetHardwareInfoVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c]), log id: 272dddc3 >2014-10-20 15:04:13,301 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-11) FINISH, GetHardwareInfoVDSCommand, log id: 272dddc3 >2014-10-20 15:04:13,322 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-8) FINISH, GetHardwareInfoVDSCommand, log id: 2625a366 >2014-10-20 15:04:13,952 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-11) Initializing Host: alma06.qa.lab.tlv.redhat.com >2014-10-20 15:04:13,981 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-8) Initializing Host: cyan-vdsg.qa.lab.tlv.redhat.com >2014-10-20 15:04:14,228 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-11) [1883f47f] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 1c39bd88-8672-408f-a593-60d6ef8f647c Type: VDS >2014-10-20 15:04:14,229 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-8) [5c27910a] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 3a2a45bd-4411-4428-aa45-a514d30d3ea6 Type: VDS >2014-10-20 15:04:16,835 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (ajp-/127.0.0.1:8702-1) Running command: LoginAdminUserCommand internal: false. >2014-10-20 15:04:16,869 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-10-20 15:04:18,518 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-32) [29188d83] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: f8f5eaee-8fd0-4b45-87db-62d61b03a916 Type: StoragePool >2014-10-20 15:04:18,548 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-32) [29188d83] Storage Pool f8f5eaee-8fd0-4b45-87db-62d61b03a916 - Updating Storage Domain a1341057-9cf8-4750-a734-656d05a5d2c5 status from Active to Unknown, reason : null >2014-10-20 15:04:18,673 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-32) [29188d83] Correlation ID: 29188d83, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center dc_35. Setting status to Non Responsive. >2014-10-20 15:04:19,008 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-32) [29188d83] hostFromVds::selectedVds - cyan-vdsg.qa.lab.tlv.redhat.com, spmStatus SPM, storage pool dc_35 >2014-10-20 15:04:19,062 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-32) [29188d83] Initialize Irs proxy from vds: 10.35.70.4 >2014-10-20 15:04:19,082 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-32) [29188d83] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Storage Pool Manager runs on Host cyan-vdsg.qa.lab.tlv.redhat.com (Address: 10.35.70.4). >2014-10-20 15:04:19,097 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to /10.35.70.4 >2014-10-20 15:04:19,301 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-12) [29188d83] START, SPMGetAllTasksInfoVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false), log id: 286a647b >2014-10-20 15:04:19,436 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-12) [29188d83] -- executeIrsBrokerCommand: Attempting on storage pool f8f5eaee-8fd0-4b45-87db-62d61b03a916 >2014-10-20 15:04:19,447 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-12) [29188d83] START, HSMGetAllTasksInfoVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6), log id: 134f6358 >2014-10-20 15:04:19,462 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-12) [29188d83] FINISH, HSMGetAllTasksInfoVDSCommand, return: [], log id: 134f6358 >2014-10-20 15:04:19,463 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-12) [29188d83] FINISH, SPMGetAllTasksInfoVDSCommand, return: [], log id: 286a647b >2014-10-20 15:04:19,463 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-7-thread-12) [29188d83] Discovered no tasks on Storage Pool dc_35 >2014-10-20 15:04:22,691 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-3) Running command: LoginUserCommand internal: false. >2014-10-20 15:04:22,760 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-10-20 15:04:45,305 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (ajp-/127.0.0.1:8702-8) [52ad9472] Lock Acquired to object EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS >, sharedLocks= ] >2014-10-20 15:04:45,409 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-7-thread-37) [52ad9472] Running command: ActivateVdsCommand internal: false. Entities affected : ID: d5b2492d-8110-4614-9ce4-4a7a118c7942 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-10-20 15:04:45,411 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-7-thread-37) [52ad9472] Before acquiring lock in order to prevent monitoring for host alma07.qa.lab.tlv.redhat.com from data-center dc_35 >2014-10-20 15:04:45,411 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-7-thread-37) [52ad9472] Lock acquired, from now a monitoring of host will be skipped for host alma07.qa.lab.tlv.redhat.com from data-center dc_35 >2014-10-20 15:04:45,450 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-37) [52ad9472] START, SetVdsStatusVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, status=Unassigned, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 7e4130fa >2014-10-20 15:04:45,495 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-37) [52ad9472] FINISH, SetVdsStatusVDSCommand, log id: 7e4130fa >2014-10-20 15:04:45,585 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-7-thread-37) [52ad9472] Activate finished. Lock released. Monitoring can run now for host alma07.qa.lab.tlv.redhat.com from data-center dc_35 >2014-10-20 15:04:45,605 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-37) [52ad9472] Correlation ID: 52ad9472, Job ID: 957efe6e-53f5-4780-b81d-4c0e7830ebdc, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com was activated by admin. >2014-10-20 15:04:45,635 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-7-thread-37) [52ad9472] Lock freed to object EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS >, sharedLocks= ] >2014-10-20 15:04:47,616 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-20 15:04:47,803 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-33) START, GetHardwareInfoVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]), log id: 5b583cef >2014-10-20 15:04:47,809 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-33) FINISH, GetHardwareInfoVDSCommand, log id: 5b583cef >2014-10-20 15:04:47,891 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-33) [20ad187c] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: d5b2492d-8110-4614-9ce4-4a7a118c7942 Type: VDS >2014-10-20 15:04:47,954 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-33) [20ad187c] START, GetHardwareInfoVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]), log id: 5119b938 >2014-10-20 15:04:47,966 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-33) [20ad187c] FINISH, GetHardwareInfoVDSCommand, log id: 5119b938 >2014-10-20 15:04:48,022 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-33) [51ab9d8e] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: f8f5eaee-8fd0-4b45-87db-62d61b03a916 Type: StoragePool >2014-10-20 15:04:48,047 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-33) [1f34ad1c] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: f8f5eaee-8fd0-4b45-87db-62d61b03a916 Type: StoragePool >2014-10-20 15:04:48,086 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-33) [1f34ad1c] START, ConnectStorageServerVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, storageType = NFS, connectionList = [{ id: 5eec92db-0201-4aa5-8598-1fe041c7a057, connection: 10.35.160.108:/RHEV/artyom_storage_rhevm_35, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 4ffed20a >2014-10-20 15:04:48,171 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-33) [1f34ad1c] FINISH, ConnectStorageServerVDSCommand, return: {5eec92db-0201-4aa5-8598-1fe041c7a057=0}, log id: 4ffed20a >2014-10-20 15:04:48,173 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-33) [1f34ad1c] Host alma07.qa.lab.tlv.redhat.com storage connection was succeeded >2014-10-20 15:04:48,222 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-7-thread-43) START, ConnectStoragePoolVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vdsId = d5b2492d-8110-4614-9ce4-4a7a118c7942, storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, masterVersion = 1), log id: 641146e8 >2014-10-20 15:04:48,504 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-7-thread-43) FINISH, ConnectStoragePoolVDSCommand, log id: 641146e8 >2014-10-20 15:04:48,524 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-33) [1f34ad1c] START, SetMOMPolicyParametersVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942), log id: 58acce84 >2014-10-20 15:04:48,585 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-33) [1f34ad1c] FINISH, SetMOMPolicyParametersVDSCommand, log id: 58acce84 >2014-10-20 15:04:48,592 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-33) [1f34ad1c] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host alma07.qa.lab.tlv.redhat.com was set to Up. >2014-10-20 15:04:48,603 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-33) [1f34ad1c] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host alma07.qa.lab.tlv.redhat.com. >2014-10-20 15:04:48,638 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-33) [75e80df5] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: d5b2492d-8110-4614-9ce4-4a7a118c7942 Type: VDS >2014-10-20 15:04:57,167 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-6) Running command: LoginUserCommand internal: false. >2014-10-20 15:04:57,178 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-10-20 15:04:58,156 INFO [org.ovirt.engine.core.bll.UpdateVmPoolWithVmsCommand] (ajp-/127.0.0.1:8702-6) [4c0813] Running command: UpdateVmPoolWithVmsCommand internal: false. Entities affected : ID: 0c2de4aa-2d90-4c80-812d-625763aa8c12 Type: VmPoolAction group EDIT_VM_POOL_CONFIGURATION with role type USER >2014-10-20 15:04:58,349 WARN [org.ovirt.engine.core.bll.AddVmAndAttachToPoolCommand] (ajp-/127.0.0.1:8702-6) [5801691b] CanDoAction of action AddVmAndAttachToPool failed. Reasons:VAR__ACTION__ADD,VAR__TYPE__VM,ACTION_TYPE_FAILED_ILLEGAL_OS_TYPE_IS_NOT_SUPPORTED_BY_ARCHITECTURE_TYPE >2014-10-20 15:04:58,405 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) [5801691b] Correlation ID: 4c0813, Job ID: c5840f71-e293-473b-bac2-dfb9a0e306dd, Call Stack: null, Custom Event ID: -1, Message: Failed to update VM Pool test_ppc_pool(User: admin). >2014-10-20 15:04:58,416 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ajp-/127.0.0.1:8702-6) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table. >2014-10-20 15:04:58,425 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (ajp-/127.0.0.1:8702-6) Operation Failed: [Cannot edit VM-Pool. Selected operating system is not supported by the architecture.] >2014-10-20 15:04:59,068 INFO [org.ovirt.engine.core.bll.aaa.LogoutBySessionCommand] (ajp-/127.0.0.1:8702-6) [55c9aec6] Running command: LogoutBySessionCommand internal: false. >2014-10-20 15:04:59,083 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-6) [59cce5fb] Running command: LogoutUserCommand internal: false. >2014-10-20 15:04:59,089 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) [59cce5fb] Correlation ID: 59cce5fb, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-10-20 15:05:25,216 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-6) Running command: LoginUserCommand internal: false. >2014-10-20 15:05:25,221 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-10-20 15:05:30,550 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-7-thread-4) [458292cc] Start running CanDoAction for command number 1/2 (Command type: RemoveVmFromPool) >2014-10-20 15:05:30,551 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-7-thread-6) [38fba992] Start running CanDoAction for command number 2/2 (Command type: RemoveVmFromPool) >2014-10-20 15:05:30,578 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-7-thread-4) [458292cc] Finish handling CanDoAction for command number 1/2 (Command type: RemoveVmFromPool) >2014-10-20 15:05:30,581 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-7-thread-6) [38fba992] Finish handling CanDoAction for command number 2/2 (Command type: RemoveVmFromPool) >2014-10-20 15:05:30,583 INFO [org.ovirt.engine.core.bll.RemoveVmFromPoolCommand] (org.ovirt.thread.pool-7-thread-5) [458292cc] Running command: RemoveVmFromPoolCommand internal: false. Entities affected : ID: 0c2de4aa-2d90-4c80-812d-625763aa8c12 Type: VmPoolAction group EDIT_VM_POOL_CONFIGURATION with role type USER >2014-10-20 15:05:30,613 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-5) [458292cc] Correlation ID: 458292cc, Call Stack: null, Custom Event ID: -1, Message: VM test_ppc_pool-2 was removed from VM Pool test_ppc_pool by admin. >2014-10-20 15:05:30,614 INFO [org.ovirt.engine.core.bll.RemoveVmFromPoolCommand] (org.ovirt.thread.pool-7-thread-5) [38fba992] Running command: RemoveVmFromPoolCommand internal: false. Entities affected : ID: 0c2de4aa-2d90-4c80-812d-625763aa8c12 Type: VmPoolAction group EDIT_VM_POOL_CONFIGURATION with role type USER >2014-10-20 15:05:30,624 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-5) [38fba992] Correlation ID: 38fba992, Call Stack: null, Custom Event ID: -1, Message: VM test_ppc_pool-1 was removed from VM Pool test_ppc_pool by admin. >2014-10-20 15:05:30,679 INFO [org.ovirt.engine.core.bll.RemoveVmPoolCommand] (org.ovirt.thread.pool-7-thread-5) [41438290] Running command: RemoveVmPoolCommand internal: true. Entities affected : ID: 0c2de4aa-2d90-4c80-812d-625763aa8c12 Type: VmPoolAction group DELETE_VM_POOL with role type USER >2014-10-20 15:05:30,763 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-5) [41438290] Correlation ID: 41438290, Call Stack: null, Custom Event ID: -1, Message: VM Pool test_ppc_pool was removed by admin. >2014-10-20 15:05:38,547 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-7-thread-22) [6f906637] Start running CanDoAction for command number 2/3 (Command type: RemoveVm) >2014-10-20 15:05:38,547 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-7-thread-20) [75e1e14b] Start running CanDoAction for command number 1/3 (Command type: RemoveVm) >2014-10-20 15:05:38,548 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-7-thread-21) [6ed777ce] Start running CanDoAction for command number 3/3 (Command type: RemoveVm) >2014-10-20 15:05:38,553 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-22) [6f906637] Lock Acquired to object EngineLock [exclusiveLocks= key: 1f557dcf-96b2-47c2-ac4a-ef4136bfc586 value: VM >, sharedLocks= ] >2014-10-20 15:05:38,556 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-20) [75e1e14b] Lock Acquired to object EngineLock [exclusiveLocks= key: 17b1e1e6-7dbe-4a3d-b556-685ca579d0e2 value: VM >, sharedLocks= ] >2014-10-20 15:05:38,553 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-21) [6ed777ce] Lock Acquired to object EngineLock [exclusiveLocks= key: e7b180d9-f84e-40f9-9d25-6a62bb713699 value: VM >, sharedLocks= ] >2014-10-20 15:05:38,631 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-7-thread-21) [6ed777ce] Finish handling CanDoAction for command number 3/3 (Command type: RemoveVm) >2014-10-20 15:05:38,638 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-7-thread-22) [6f906637] Finish handling CanDoAction for command number 2/3 (Command type: RemoveVm) >2014-10-20 15:05:38,658 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-7-thread-20) [75e1e14b] Finish handling CanDoAction for command number 1/3 (Command type: RemoveVm) >2014-10-20 15:05:38,684 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-23) [75e1e14b] Running command: RemoveVmCommand internal: false. Entities affected : ID: 17b1e1e6-7dbe-4a3d-b556-685ca579d0e2 Type: VMAction group DELETE_VM with role type USER >2014-10-20 15:05:38,690 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-23) [75e1e14b] START, SetVmStatusVDSCommand( vmId = 17b1e1e6-7dbe-4a3d-b556-685ca579d0e2, status = ImageLocked, exit status = Normal), log id: 3ed7332d >2014-10-20 15:05:38,705 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-23) [75e1e14b] FINISH, SetVmStatusVDSCommand, log id: 3ed7332d >2014-10-20 15:05:38,710 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-23) [75e1e14b] Lock freed to object EngineLock [exclusiveLocks= key: 17b1e1e6-7dbe-4a3d-b556-685ca579d0e2 value: VM >, sharedLocks= ] >2014-10-20 15:05:38,849 INFO [org.ovirt.engine.core.bll.RemoveAllVmImagesCommand] (org.ovirt.thread.pool-7-thread-23) [447718d8] Running command: RemoveAllVmImagesCommand internal: true. Entities affected : ID: 17b1e1e6-7dbe-4a3d-b556-685ca579d0e2 Type: VM >2014-10-20 15:05:38,908 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-7-thread-23) [61338607] Running command: RemoveImageCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage >2014-10-20 15:05:39,236 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-7-thread-23) [61338607] START, DeleteImageGroupVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = 40e2fdae-cc93-4242-aa96-1624d7bb788e, postZeros = false, forceDelete = false), log id: b9775ed >2014-10-20 15:05:39,637 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-7-thread-23) [61338607] FINISH, DeleteImageGroupVDSCommand, log id: b9775ed >2014-10-20 15:05:39,651 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-23) [61338607] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 874fbb7f-f4d1-421a-a6c1-303ae0c0795d >2014-10-20 15:05:39,651 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-7-thread-23) [61338607] CommandMultiAsyncTasks::AttachTask: Attaching task 4007512b-5286-4bd9-83fe-ea343245420c to command 874fbb7f-f4d1-421a-a6c1-303ae0c0795d. >2014-10-20 15:05:39,686 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-7-thread-23) [61338607] Adding task 4007512b-5286-4bd9-83fe-ea343245420c (Parent Command RemoveVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-10-20 15:05:40,012 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-23) [61338607] Correlation ID: 75e1e14b, Job ID: b2b910cb-08b7-4506-9f03-db39c3ce3e3a, Call Stack: null, Custom Event ID: -1, Message: VM ppc_vm was successfully removed. >2014-10-20 15:05:40,013 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-23) [61338607] BaseAsyncTask::startPollingTask: Starting to poll task 4007512b-5286-4bd9-83fe-ea343245420c. >2014-10-20 15:05:40,045 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-23) [6f906637] Running command: RemoveVmCommand internal: false. Entities affected : ID: 1f557dcf-96b2-47c2-ac4a-ef4136bfc586 Type: VMAction group DELETE_VM with role type USER >2014-10-20 15:05:40,046 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-23) [6f906637] START, SetVmStatusVDSCommand( vmId = 1f557dcf-96b2-47c2-ac4a-ef4136bfc586, status = ImageLocked, exit status = Normal), log id: a5718e5 >2014-10-20 15:05:40,051 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-23) [6f906637] FINISH, SetVmStatusVDSCommand, log id: a5718e5 >2014-10-20 15:05:40,055 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-23) [6f906637] Lock freed to object EngineLock [exclusiveLocks= key: 1f557dcf-96b2-47c2-ac4a-ef4136bfc586 value: VM >, sharedLocks= ] >2014-10-20 15:05:40,139 INFO [org.ovirt.engine.core.bll.RemoveAllVmImagesCommand] (org.ovirt.thread.pool-7-thread-23) [7bca0fdb] Running command: RemoveAllVmImagesCommand internal: true. Entities affected : ID: 1f557dcf-96b2-47c2-ac4a-ef4136bfc586 Type: VM >2014-10-20 15:05:40,143 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-7-thread-23) [4d06aaaa] Running command: RemoveImageCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage >2014-10-20 15:05:40,171 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-7-thread-23) [4d06aaaa] START, DeleteImageGroupVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = daa95a9c-26c4-4324-a7ce-c86c5085bee1, postZeros = false, forceDelete = false), log id: 2c4c83f8 >2014-10-20 15:05:40,343 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-7-thread-23) [4d06aaaa] FINISH, DeleteImageGroupVDSCommand, log id: 2c4c83f8 >2014-10-20 15:05:40,348 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-23) [4d06aaaa] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 763838bd-f1ab-45ec-bb95-d3d82cf8f295 >2014-10-20 15:05:40,349 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-7-thread-23) [4d06aaaa] CommandMultiAsyncTasks::AttachTask: Attaching task a55c5398-92c7-4231-99c7-5465c03d937c to command 763838bd-f1ab-45ec-bb95-d3d82cf8f295. >2014-10-20 15:05:40,370 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-7-thread-23) [4d06aaaa] Adding task a55c5398-92c7-4231-99c7-5465c03d937c (Parent Command RemoveVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-10-20 15:05:40,632 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-23) [4d06aaaa] Correlation ID: 6f906637, Job ID: 1b17e77a-6499-4e89-88a4-972ff21d1507, Call Stack: null, Custom Event ID: -1, Message: VM test_ppc_pool-1 was successfully removed. >2014-10-20 15:05:40,632 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-23) [4d06aaaa] BaseAsyncTask::startPollingTask: Starting to poll task a55c5398-92c7-4231-99c7-5465c03d937c. >2014-10-20 15:05:40,653 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-23) [6ed777ce] Running command: RemoveVmCommand internal: false. Entities affected : ID: e7b180d9-f84e-40f9-9d25-6a62bb713699 Type: VMAction group DELETE_VM with role type USER >2014-10-20 15:05:40,658 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-23) [6ed777ce] START, SetVmStatusVDSCommand( vmId = e7b180d9-f84e-40f9-9d25-6a62bb713699, status = ImageLocked, exit status = Normal), log id: 4b9f8968 >2014-10-20 15:05:40,663 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-23) [6ed777ce] FINISH, SetVmStatusVDSCommand, log id: 4b9f8968 >2014-10-20 15:05:40,665 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-23) [6ed777ce] Lock freed to object EngineLock [exclusiveLocks= key: e7b180d9-f84e-40f9-9d25-6a62bb713699 value: VM >, sharedLocks= ] >2014-10-20 15:05:40,682 INFO [org.ovirt.engine.core.bll.RemoveAllVmImagesCommand] (org.ovirt.thread.pool-7-thread-23) [6df742a6] Running command: RemoveAllVmImagesCommand internal: true. Entities affected : ID: e7b180d9-f84e-40f9-9d25-6a62bb713699 Type: VM >2014-10-20 15:05:40,697 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-7-thread-23) [2602ad90] Running command: RemoveImageCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage >2014-10-20 15:05:40,718 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-7-thread-23) [2602ad90] START, DeleteImageGroupVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = 45206a2d-81c8-43fe-a2f7-8ffee8f6c96c, postZeros = false, forceDelete = false), log id: 1c0ea26c >2014-10-20 15:05:40,887 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-7-thread-23) [2602ad90] FINISH, DeleteImageGroupVDSCommand, log id: 1c0ea26c >2014-10-20 15:05:40,893 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-23) [2602ad90] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command a1d00deb-d379-4c7b-b0d0-963462e9a0fc >2014-10-20 15:05:40,893 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-7-thread-23) [2602ad90] CommandMultiAsyncTasks::AttachTask: Attaching task 5be33937-5b1a-4f64-b0ad-a8405163f2af to command a1d00deb-d379-4c7b-b0d0-963462e9a0fc. >2014-10-20 15:05:40,906 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-7-thread-23) [2602ad90] Adding task 5be33937-5b1a-4f64-b0ad-a8405163f2af (Parent Command RemoveVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-10-20 15:05:40,975 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-23) [2602ad90] Correlation ID: 6ed777ce, Job ID: d7893d37-32f8-4aa2-aa73-57f461e62b94, Call Stack: null, Custom Event ID: -1, Message: VM test_ppc_pool-2 was successfully removed. >2014-10-20 15:05:40,976 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-23) [2602ad90] BaseAsyncTask::startPollingTask: Starting to poll task 5be33937-5b1a-4f64-b0ad-a8405163f2af. >2014-10-20 15:05:47,457 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-62) Polling and updating Async Tasks: 3 tasks, 3 tasks to poll now >2014-10-20 15:05:47,534 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-62) SPMAsyncTask::PollTask: Polling task 4007512b-5286-4bd9-83fe-ea343245420c (Parent Command RemoveVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-10-20 15:05:47,548 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-62) BaseAsyncTask::onTaskEndSuccess: Task 4007512b-5286-4bd9-83fe-ea343245420c (Parent Command RemoveVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-10-20 15:05:47,549 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-62) CommandAsyncTask::endActionIfNecessary: All tasks of command 874fbb7f-f4d1-421a-a6c1-303ae0c0795d has ended -> executing endAction >2014-10-20 15:05:47,552 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-62) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: 874fbb7f-f4d1-421a-a6c1-303ae0c0795d): calling endAction . >2014-10-20 15:05:47,552 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-62) SPMAsyncTask::PollTask: Polling task 5be33937-5b1a-4f64-b0ad-a8405163f2af (Parent Command RemoveVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-10-20 15:05:47,553 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-40) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction RemoveVm, executionIndex: 0 >2014-10-20 15:05:47,564 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-62) BaseAsyncTask::onTaskEndSuccess: Task 5be33937-5b1a-4f64-b0ad-a8405163f2af (Parent Command RemoveVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-10-20 15:05:47,565 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-62) CommandAsyncTask::endActionIfNecessary: All tasks of command a1d00deb-d379-4c7b-b0d0-963462e9a0fc has ended -> executing endAction >2014-10-20 15:05:47,565 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-62) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: a1d00deb-d379-4c7b-b0d0-963462e9a0fc): calling endAction . >2014-10-20 15:05:47,566 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-62) SPMAsyncTask::PollTask: Polling task a55c5398-92c7-4231-99c7-5465c03d937c (Parent Command RemoveVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-10-20 15:05:47,566 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-39) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction RemoveVm, executionIndex: 0 >2014-10-20 15:05:47,572 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-62) BaseAsyncTask::onTaskEndSuccess: Task a55c5398-92c7-4231-99c7-5465c03d937c (Parent Command RemoveVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-10-20 15:05:47,577 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-62) CommandAsyncTask::endActionIfNecessary: All tasks of command 763838bd-f1ab-45ec-bb95-d3d82cf8f295 has ended -> executing endAction >2014-10-20 15:05:47,578 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-62) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: 763838bd-f1ab-45ec-bb95-d3d82cf8f295): calling endAction . >2014-10-20 15:05:47,579 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-41) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction RemoveVm, executionIndex: 0 >2014-10-20 15:05:47,617 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-39) Ending command successfully: org.ovirt.engine.core.bll.RemoveVmCommand >2014-10-20 15:05:47,618 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-40) Ending command successfully: org.ovirt.engine.core.bll.RemoveVmCommand >2014-10-20 15:05:47,621 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-40) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RemoveVm completed, handling the result. >2014-10-20 15:05:47,621 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-40) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RemoveVm succeeded, clearing tasks. >2014-10-20 15:05:47,636 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-41) Ending command successfully: org.ovirt.engine.core.bll.RemoveVmCommand >2014-10-20 15:05:47,638 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-41) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RemoveVm completed, handling the result. >2014-10-20 15:05:47,638 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-41) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RemoveVm succeeded, clearing tasks. >2014-10-20 15:05:47,643 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-40) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 4007512b-5286-4bd9-83fe-ea343245420c >2014-10-20 15:05:47,644 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-41) SPMAsyncTask::ClearAsyncTask: Attempting to clear task a55c5398-92c7-4231-99c7-5465c03d937c >2014-10-20 15:05:47,646 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-39) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RemoveVm completed, handling the result. >2014-10-20 15:05:47,646 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-39) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RemoveVm succeeded, clearing tasks. >2014-10-20 15:05:47,645 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-40) START, SPMClearTaskVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, taskId = 4007512b-5286-4bd9-83fe-ea343245420c), log id: 13fba683 >2014-10-20 15:05:47,652 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-41) START, SPMClearTaskVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, taskId = a55c5398-92c7-4231-99c7-5465c03d937c), log id: 753b7ace >2014-10-20 15:05:47,654 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-39) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 5be33937-5b1a-4f64-b0ad-a8405163f2af >2014-10-20 15:05:47,654 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-39) START, SPMClearTaskVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, taskId = 5be33937-5b1a-4f64-b0ad-a8405163f2af), log id: 7a242315 >2014-10-20 15:05:47,676 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-40) START, HSMClearTaskVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, taskId=4007512b-5286-4bd9-83fe-ea343245420c), log id: 52eb5ea0 >2014-10-20 15:05:47,744 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-40) FINISH, HSMClearTaskVDSCommand, log id: 52eb5ea0 >2014-10-20 15:05:47,744 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-40) FINISH, SPMClearTaskVDSCommand, log id: 13fba683 >2014-10-20 15:05:47,758 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-39) START, HSMClearTaskVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, taskId=5be33937-5b1a-4f64-b0ad-a8405163f2af), log id: 294605e4 >2014-10-20 15:05:47,795 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-40) BaseAsyncTask::removeTaskFromDB: Removed task 4007512b-5286-4bd9-83fe-ea343245420c from DataBase >2014-10-20 15:05:47,796 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-40) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 874fbb7f-f4d1-421a-a6c1-303ae0c0795d >2014-10-20 15:05:47,796 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-39) FINISH, HSMClearTaskVDSCommand, log id: 294605e4 >2014-10-20 15:05:47,796 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-39) FINISH, SPMClearTaskVDSCommand, log id: 7a242315 >2014-10-20 15:05:47,798 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-41) START, HSMClearTaskVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, taskId=a55c5398-92c7-4231-99c7-5465c03d937c), log id: 55fa684d >2014-10-20 15:05:47,820 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-39) BaseAsyncTask::removeTaskFromDB: Removed task 5be33937-5b1a-4f64-b0ad-a8405163f2af from DataBase >2014-10-20 15:05:47,820 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-39) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity a1d00deb-d379-4c7b-b0d0-963462e9a0fc >2014-10-20 15:05:47,832 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-41) FINISH, HSMClearTaskVDSCommand, log id: 55fa684d >2014-10-20 15:05:47,832 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-41) FINISH, SPMClearTaskVDSCommand, log id: 753b7ace >2014-10-20 15:05:47,860 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-41) BaseAsyncTask::removeTaskFromDB: Removed task a55c5398-92c7-4231-99c7-5465c03d937c from DataBase >2014-10-20 15:05:47,861 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-41) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 763838bd-f1ab-45ec-bb95-d3d82cf8f295 >2014-10-20 15:06:00,575 INFO [org.ovirt.engine.core.bll.AddVmFromScratchCommand] (ajp-/127.0.0.1:8702-9) [7027b1cc] Lock Acquired to object EngineLock [exclusiveLocks= key: ppc_vm value: VM_NAME >, sharedLocks= ] >2014-10-20 15:06:00,645 INFO [org.ovirt.engine.core.bll.AddVmFromScratchCommand] (ajp-/127.0.0.1:8702-9) [7027b1cc] Running command: AddVmFromScratchCommand internal: false. Entities affected : ID: 63702093-c04f-417c-8045-aa56334b54ee Type: VdsGroupsAction group CREATE_VM with role type USER >2014-10-20 15:06:00,842 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [7027b1cc] Correlation ID: 7027b1cc, Job ID: 10da805b-9b00-4c7e-9a7c-3390ad8bfee1, Call Stack: null, Custom Event ID: -1, Message: VM ppc_vm was created by admin. >2014-10-20 15:06:00,846 INFO [org.ovirt.engine.core.bll.AddVmFromScratchCommand] (ajp-/127.0.0.1:8702-9) [7027b1cc] Lock freed to object EngineLock [exclusiveLocks= key: ppc_vm value: VM_NAME >, sharedLocks= ] >2014-10-20 15:06:02,301 INFO [org.ovirt.engine.core.bll.network.vm.AddVmInterfaceCommand] (ajp-/127.0.0.1:8702-7) [51190ca0] Running command: AddVmInterfaceCommand internal: false. Entities affected : ID: 3c71926d-02cd-46fa-8975-f0986ca49674 Type: VMAction group CONFIGURE_VM_NETWORK with role type USER, ID: c078ca43-dc90-4ffe-a17e-68721438ee95 Type: VnicProfileAction group CONFIGURE_VM_NETWORK with role type USER >2014-10-20 15:06:02,462 INFO [org.ovirt.engine.core.bll.network.vm.ActivateDeactivateVmNicCommand] (ajp-/127.0.0.1:8702-7) [3faa7430] Running command: ActivateDeactivateVmNicCommand internal: true. Entities affected : ID: 3c71926d-02cd-46fa-8975-f0986ca49674 Type: VM >2014-10-20 15:06:02,520 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [3faa7430] Correlation ID: 3faa7430, Call Stack: null, Custom Event ID: -1, Message: Network Interface nic1 (VirtIO) was plugged to VM ppc_vm. (User: admin) >2014-10-20 15:06:02,679 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [3faa7430] Correlation ID: 51190ca0, Call Stack: null, Custom Event ID: -1, Message: Interface nic1 (VirtIO) was added to VM ppc_vm. (User: admin) >2014-10-20 15:06:02,761 INFO [org.ovirt.engine.core.bll.network.vm.ReorderVmNicsCommand] (ajp-/127.0.0.1:8702-5) [5a09fff0] Running command: ReorderVmNicsCommand internal: false. Entities affected : ID: 3c71926d-02cd-46fa-8975-f0986ca49674 Type: VMAction group CREATE_VM with role type USER >2014-10-20 15:06:14,340 INFO [org.ovirt.engine.core.bll.AddDiskCommand] (ajp-/127.0.0.1:8702-11) [fde0cfe] Lock Acquired to object EngineLock [exclusiveLocks= key: 3c71926d-02cd-46fa-8975-f0986ca49674 value: VM_DISK_BOOT >, sharedLocks= key: 3c71926d-02cd-46fa-8975-f0986ca49674 value: VM >] >2014-10-20 15:06:14,383 INFO [org.ovirt.engine.core.bll.AddDiskCommand] (ajp-/127.0.0.1:8702-11) [fde0cfe] Running command: AddDiskCommand internal: false. Entities affected : ID: 3c71926d-02cd-46fa-8975-f0986ca49674 Type: VMAction group CONFIGURE_VM_STORAGE with role type USER, ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: StorageAction group CREATE_DISK with role type USER >2014-10-20 15:06:14,446 INFO [org.ovirt.engine.core.bll.AddImageFromScratchCommand] (ajp-/127.0.0.1:8702-11) [69e2ca7d] Running command: AddImageFromScratchCommand internal: true. Entities affected : ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: Storage >2014-10-20 15:06:14,502 INFO [org.ovirt.engine.core.bll.AddImageFromScratchCommand] (ajp-/127.0.0.1:8702-11) [69e2ca7d] Lock freed to object EngineLock [exclusiveLocks= key: 3c71926d-02cd-46fa-8975-f0986ca49674 value: VM_DISK_BOOT >, sharedLocks= key: 3c71926d-02cd-46fa-8975-f0986ca49674 value: VM >] >2014-10-20 15:06:14,504 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (ajp-/127.0.0.1:8702-11) [69e2ca7d] START, CreateImageVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = 4323cfe2-a20f-48d0-8d3c-d02e52d43625, imageSizeInBytes = 1073741824, volumeFormat = RAW, newImageId = 4b471ed5-62c7-44d1-a683-0d86117075da, newImageDescription = ), log id: 4f1ad960 >2014-10-20 15:06:14,506 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (ajp-/127.0.0.1:8702-11) [69e2ca7d] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-10-20 15:06:14,654 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (ajp-/127.0.0.1:8702-11) [69e2ca7d] FINISH, CreateImageVDSCommand, return: 4b471ed5-62c7-44d1-a683-0d86117075da, log id: 4f1ad960 >2014-10-20 15:06:14,661 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (ajp-/127.0.0.1:8702-11) [69e2ca7d] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 19ec7ec2-fdaa-4fac-b986-375ce73f251d >2014-10-20 15:06:14,661 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (ajp-/127.0.0.1:8702-11) [69e2ca7d] CommandMultiAsyncTasks::AttachTask: Attaching task 71ef7468-26ca-4c11-853f-f29ce54f503e to command 19ec7ec2-fdaa-4fac-b986-375ce73f251d. >2014-10-20 15:06:14,689 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (ajp-/127.0.0.1:8702-11) [69e2ca7d] Adding task 71ef7468-26ca-4c11-853f-f29ce54f503e (Parent Command AddDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-10-20 15:06:14,766 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) [69e2ca7d] Correlation ID: fde0cfe, Job ID: e1ffcb54-e10f-4e12-af2a-565a85473423, Call Stack: null, Custom Event ID: -1, Message: Add-Disk operation of ppc_vm_Disk1 was initiated on VM ppc_vm by admin. >2014-10-20 15:06:14,768 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (ajp-/127.0.0.1:8702-11) [69e2ca7d] BaseAsyncTask::startPollingTask: Starting to poll task 71ef7468-26ca-4c11-853f-f29ce54f503e. >2014-10-20 15:06:17,587 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-74) Polling and updating Async Tasks: 4 tasks, 1 tasks to poll now >2014-10-20 15:06:17,679 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-74) SPMAsyncTask::PollTask: Polling task 71ef7468-26ca-4c11-853f-f29ce54f503e (Parent Command AddDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-10-20 15:06:17,722 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-74) BaseAsyncTask::onTaskEndSuccess: Task 71ef7468-26ca-4c11-853f-f29ce54f503e (Parent Command AddDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-10-20 15:06:17,724 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-74) CommandAsyncTask::endActionIfNecessary: All tasks of command 19ec7ec2-fdaa-4fac-b986-375ce73f251d has ended -> executing endAction >2014-10-20 15:06:17,724 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-74) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: 19ec7ec2-fdaa-4fac-b986-375ce73f251d): calling endAction . >2014-10-20 15:06:17,725 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-31) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction AddDisk, executionIndex: 0 >2014-10-20 15:06:17,758 INFO [org.ovirt.engine.core.bll.AddDiskCommand] (org.ovirt.thread.pool-7-thread-31) [fde0cfe] Ending command successfully: org.ovirt.engine.core.bll.AddDiskCommand >2014-10-20 15:06:17,969 INFO [org.ovirt.engine.core.bll.AddImageFromScratchCommand] (org.ovirt.thread.pool-7-thread-31) [69e2ca7d] Ending command successfully: org.ovirt.engine.core.bll.AddImageFromScratchCommand >2014-10-20 15:06:17,985 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-31) [69e2ca7d] START, GetImageInfoVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = 4323cfe2-a20f-48d0-8d3c-d02e52d43625, imageId = 4b471ed5-62c7-44d1-a683-0d86117075da), log id: 78d07ecb >2014-10-20 15:06:18,038 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-31) [69e2ca7d] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@e64a7e83, log id: 78d07ecb >2014-10-20 15:06:18,096 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-31) Correlation ID: fde0cfe, Job ID: e1ffcb54-e10f-4e12-af2a-565a85473423, Call Stack: null, Custom Event ID: -1, Message: The disk ppc_vm_Disk1 was successfully added to VM ppc_vm. >2014-10-20 15:06:18,096 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-31) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddDisk completed, handling the result. >2014-10-20 15:06:18,096 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-31) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddDisk succeeded, clearing tasks. >2014-10-20 15:06:18,100 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-31) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 71ef7468-26ca-4c11-853f-f29ce54f503e >2014-10-20 15:06:18,100 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-31) START, SPMClearTaskVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, taskId = 71ef7468-26ca-4c11-853f-f29ce54f503e), log id: 769528e >2014-10-20 15:06:18,105 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-31) START, HSMClearTaskVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, taskId=71ef7468-26ca-4c11-853f-f29ce54f503e), log id: 13aaec6f >2014-10-20 15:06:18,142 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-31) FINISH, HSMClearTaskVDSCommand, log id: 13aaec6f >2014-10-20 15:06:18,142 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-31) FINISH, SPMClearTaskVDSCommand, log id: 769528e >2014-10-20 15:06:18,147 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-31) BaseAsyncTask::removeTaskFromDB: Removed task 71ef7468-26ca-4c11-853f-f29ce54f503e from DataBase >2014-10-20 15:06:18,147 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-31) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 19ec7ec2-fdaa-4fac-b986-375ce73f251d >2014-10-20 15:06:22,861 INFO [org.ovirt.engine.core.bll.RemoveVmTemplateCommand] (ajp-/127.0.0.1:8702-7) [a117b54] Lock Acquired to object EngineLock [exclusiveLocks= key: e246188b-771c-4107-9dbf-8a61c552be0f value: TEMPLATE >, sharedLocks= ] >2014-10-20 15:06:22,970 INFO [org.ovirt.engine.core.bll.RemoveVmTemplateCommand] (org.ovirt.thread.pool-7-thread-38) [a117b54] Running command: RemoveVmTemplateCommand internal: false. Entities affected : ID: e246188b-771c-4107-9dbf-8a61c552be0f Type: VmTemplateAction group DELETE_TEMPLATE with role type USER >2014-10-20 15:06:22,995 INFO [org.ovirt.engine.core.bll.RemoveAllVmTemplateImageTemplatesCommand] (org.ovirt.thread.pool-7-thread-38) [a117b54] Running command: RemoveAllVmTemplateImageTemplatesCommand internal: true. Entities affected : ID: e246188b-771c-4107-9dbf-8a61c552be0f Type: VmTemplate >2014-10-20 15:06:23,018 INFO [org.ovirt.engine.core.bll.RemoveTemplateSnapshotCommand] (org.ovirt.thread.pool-7-thread-38) [1b72a93] Running command: RemoveTemplateSnapshotCommand internal: true. Entities affected : ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: Storage >2014-10-20 15:06:23,097 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-7-thread-38) [1b72a93] START, DeleteImageGroupVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = a217a5af-cb04-41e4-9252-e82aed18c789, postZeros = false, forceDelete = false), log id: 511243a8 >2014-10-20 15:06:23,264 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-7-thread-38) [1b72a93] FINISH, DeleteImageGroupVDSCommand, log id: 511243a8 >2014-10-20 15:06:23,277 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-38) [1b72a93] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 07086ce8-df66-4fde-ba0b-25cae17a097e >2014-10-20 15:06:23,277 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-7-thread-38) [1b72a93] CommandMultiAsyncTasks::AttachTask: Attaching task a6ff99c0-9f43-448c-8330-60467b3dfa8a to command 07086ce8-df66-4fde-ba0b-25cae17a097e. >2014-10-20 15:06:23,289 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-7-thread-38) [1b72a93] Adding task a6ff99c0-9f43-448c-8330-60467b3dfa8a (Parent Command RemoveVmTemplate, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-10-20 15:06:23,368 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-38) [1b72a93] Correlation ID: a117b54, Job ID: 814b6543-9a6e-4cad-90bc-058b0bc160ab, Call Stack: null, Custom Event ID: -1, Message: Removal of Template clean_ppc_template was initiated by admin. >2014-10-20 15:06:23,368 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-38) [1b72a93] BaseAsyncTask::startPollingTask: Starting to poll task a6ff99c0-9f43-448c-8330-60467b3dfa8a. >2014-10-20 15:06:27,726 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-17) Polling and updating Async Tasks: 5 tasks, 1 tasks to poll now >2014-10-20 15:06:27,739 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-17) SPMAsyncTask::PollTask: Polling task a6ff99c0-9f43-448c-8330-60467b3dfa8a (Parent Command RemoveVmTemplate, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-10-20 15:06:27,759 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-17) BaseAsyncTask::onTaskEndSuccess: Task a6ff99c0-9f43-448c-8330-60467b3dfa8a (Parent Command RemoveVmTemplate, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-10-20 15:06:27,761 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-17) CommandAsyncTask::endActionIfNecessary: All tasks of command 07086ce8-df66-4fde-ba0b-25cae17a097e has ended -> executing endAction >2014-10-20 15:06:27,761 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-17) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: 07086ce8-df66-4fde-ba0b-25cae17a097e): calling endAction . >2014-10-20 15:06:27,762 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-45) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction RemoveVmTemplate, executionIndex: 0 >2014-10-20 15:06:27,813 INFO [org.ovirt.engine.core.bll.RemoveVmTemplateCommand] (org.ovirt.thread.pool-7-thread-45) Ending command successfully: org.ovirt.engine.core.bll.RemoveVmTemplateCommand >2014-10-20 15:06:27,842 INFO [org.ovirt.engine.core.bll.RemoveVmTemplateCommand] (org.ovirt.thread.pool-7-thread-45) Lock freed to object EngineLock [exclusiveLocks= key: e246188b-771c-4107-9dbf-8a61c552be0f value: TEMPLATE >, sharedLocks= ] >2014-10-20 15:06:27,858 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-45) Correlation ID: a117b54, Call Stack: null, Custom Event ID: -1, Message: Removal of Template clean_ppc_template has been completed. >2014-10-20 15:06:27,858 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-45) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RemoveVmTemplate completed, handling the result. >2014-10-20 15:06:27,858 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-45) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RemoveVmTemplate succeeded, clearing tasks. >2014-10-20 15:06:27,862 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-45) SPMAsyncTask::ClearAsyncTask: Attempting to clear task a6ff99c0-9f43-448c-8330-60467b3dfa8a >2014-10-20 15:06:27,863 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-45) START, SPMClearTaskVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, taskId = a6ff99c0-9f43-448c-8330-60467b3dfa8a), log id: 6090f0f9 >2014-10-20 15:06:27,865 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-45) START, HSMClearTaskVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, taskId=a6ff99c0-9f43-448c-8330-60467b3dfa8a), log id: 4bbc9cf6 >2014-10-20 15:06:27,904 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-45) FINISH, HSMClearTaskVDSCommand, log id: 4bbc9cf6 >2014-10-20 15:06:27,904 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-45) FINISH, SPMClearTaskVDSCommand, log id: 6090f0f9 >2014-10-20 15:06:27,926 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-45) BaseAsyncTask::removeTaskFromDB: Removed task a6ff99c0-9f43-448c-8330-60467b3dfa8a from DataBase >2014-10-20 15:06:27,926 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-45) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 07086ce8-df66-4fde-ba0b-25cae17a097e >2014-10-20 15:06:44,820 INFO [org.ovirt.engine.core.bll.AddVmTemplateCommand] (ajp-/127.0.0.1:8702-5) Command 39ca3782-12c3-42f7-b190-803b1694898d persisting async task placeholder for child command c1d89692-475e-4461-b545-ed89e49b3451 >2014-10-20 15:06:45,012 INFO [org.ovirt.engine.core.bll.AddVmTemplateCommand] (ajp-/127.0.0.1:8702-5) Running command: AddVmTemplateCommand internal: false. Entities affected : ID: f8f5eaee-8fd0-4b45-87db-62d61b03a916 Type: StoragePoolAction group CREATE_TEMPLATE with role type USER >2014-10-20 15:06:45,014 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (ajp-/127.0.0.1:8702-5) START, SetVmStatusVDSCommand( vmId = 3c71926d-02cd-46fa-8975-f0986ca49674, status = ImageLocked, exit status = Normal), log id: 7f24a0ae >2014-10-20 15:06:45,020 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (ajp-/127.0.0.1:8702-5) FINISH, SetVmStatusVDSCommand, log id: 7f24a0ae >2014-10-20 15:06:45,066 INFO [org.ovirt.engine.core.bll.CreateImageTemplateCommand] (ajp-/127.0.0.1:8702-5) [7215f849] Running command: CreateImageTemplateCommand internal: true. Entities affected : ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: Storage >2014-10-20 15:06:45,086 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (ajp-/127.0.0.1:8702-5) [7215f849] START, CopyImageVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = 4323cfe2-a20f-48d0-8d3c-d02e52d43625, imageId = 4b471ed5-62c7-44d1-a683-0d86117075da, dstImageGroupId = 3778d539-c7de-48fc-be30-9a02bf87665a, vmId = 3c71926d-02cd-46fa-8975-f0986ca49674, dstImageId = a9c8a68a-7a40-4b35-95e8-b8f8c8d67057, imageDescription = Active VM, dstStorageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, copyVolumeType = SharedVol, volumeFormat = RAW, preallocate = Sparse, postZero = false, force = false), log id: 713ae4dc >2014-10-20 15:06:45,087 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (ajp-/127.0.0.1:8702-5) [7215f849] -- executeIrsBrokerCommand: calling 'copyImage' with two new parameters: description and UUID >2014-10-20 15:06:45,088 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (ajp-/127.0.0.1:8702-5) [7215f849] -- copyImage parameters: > sdUUID=a1341057-9cf8-4750-a734-656d05a5d2c5 > spUUID=f8f5eaee-8fd0-4b45-87db-62d61b03a916 > vmGUID=3c71926d-02cd-46fa-8975-f0986ca49674 > srcImageGUID=4323cfe2-a20f-48d0-8d3c-d02e52d43625 > srcVolUUID=4b471ed5-62c7-44d1-a683-0d86117075da > dstImageGUID=3778d539-c7de-48fc-be30-9a02bf87665a > dstVolUUID=a9c8a68a-7a40-4b35-95e8-b8f8c8d67057 > descr=Active VM > dstSdUUID=a1341057-9cf8-4750-a734-656d05a5d2c5 >2014-10-20 15:06:45,438 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (ajp-/127.0.0.1:8702-5) [7215f849] FINISH, CopyImageVDSCommand, return: 00000000-0000-0000-0000-000000000000, log id: 713ae4dc >2014-10-20 15:06:45,445 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (ajp-/127.0.0.1:8702-5) [7215f849] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 39ca3782-12c3-42f7-b190-803b1694898d >2014-10-20 15:06:45,445 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (ajp-/127.0.0.1:8702-5) [7215f849] CommandMultiAsyncTasks::AttachTask: Attaching task ba1faaa1-e56a-4aca-8c32-abcc38304491 to command 39ca3782-12c3-42f7-b190-803b1694898d. >2014-10-20 15:06:45,456 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (ajp-/127.0.0.1:8702-5) [7215f849] Adding task ba1faaa1-e56a-4aca-8c32-abcc38304491 (Parent Command AddVmTemplate, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-10-20 15:06:45,604 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) Correlation ID: 1975feb8, Job ID: 3b96064e-b712-42a6-9fbd-14f3a7315589, Call Stack: null, Custom Event ID: -1, Message: Creation of Template clean_ppc_template from VM ppc_vm was initiated by admin. >2014-10-20 15:06:45,604 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (ajp-/127.0.0.1:8702-5) BaseAsyncTask::startPollingTask: Starting to poll task ba1faaa1-e56a-4aca-8c32-abcc38304491. >2014-10-20 15:06:47,762 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-86) Polling and updating Async Tasks: 6 tasks, 1 tasks to poll now >2014-10-20 15:06:47,790 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-86) SPMAsyncTask::PollTask: Polling task ba1faaa1-e56a-4aca-8c32-abcc38304491 (Parent Command AddVmTemplate, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status running. >2014-10-20 15:06:47,790 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-86) Finished polling Tasks, will poll again in 10 seconds. >2014-10-20 15:07:07,444 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-57) Setting new tasks map. The map contains now 3 tasks >2014-10-20 15:07:07,813 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-61) Polling and updating Async Tasks: 3 tasks, 1 tasks to poll now >2014-10-20 15:07:07,826 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-61) SPMAsyncTask::PollTask: Polling task ba1faaa1-e56a-4aca-8c32-abcc38304491 (Parent Command AddVmTemplate, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-10-20 15:07:07,831 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-61) BaseAsyncTask::onTaskEndSuccess: Task ba1faaa1-e56a-4aca-8c32-abcc38304491 (Parent Command AddVmTemplate, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-10-20 15:07:07,832 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-61) CommandAsyncTask::endActionIfNecessary: All tasks of command 39ca3782-12c3-42f7-b190-803b1694898d has ended -> executing endAction >2014-10-20 15:07:07,832 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-61) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: 39ca3782-12c3-42f7-b190-803b1694898d): calling endAction . >2014-10-20 15:07:07,832 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-1) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction AddVmTemplate, executionIndex: 0 >2014-10-20 15:07:07,952 INFO [org.ovirt.engine.core.bll.AddVmTemplateCommand] (org.ovirt.thread.pool-7-thread-1) [1975feb8] Ending command successfully: org.ovirt.engine.core.bll.AddVmTemplateCommand >2014-10-20 15:07:07,960 INFO [org.ovirt.engine.core.bll.CreateImageTemplateCommand] (org.ovirt.thread.pool-7-thread-1) [7215f849] Ending command successfully: org.ovirt.engine.core.bll.CreateImageTemplateCommand >2014-10-20 15:07:07,975 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-1) [7215f849] START, GetImageInfoVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = 3778d539-c7de-48fc-be30-9a02bf87665a, imageId = a9c8a68a-7a40-4b35-95e8-b8f8c8d67057), log id: b2a5208 >2014-10-20 15:07:08,015 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-1) [7215f849] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@d0c5196, log id: b2a5208 >2014-10-20 15:07:08,024 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-1) START, SetVmStatusVDSCommand( vmId = 3c71926d-02cd-46fa-8975-f0986ca49674, status = Down, exit status = Normal), log id: 25816e7b >2014-10-20 15:07:08,028 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-1) FINISH, SetVmStatusVDSCommand, log id: 25816e7b >2014-10-20 15:07:08,066 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-1) Correlation ID: 1975feb8, Job ID: 3b96064e-b712-42a6-9fbd-14f3a7315589, Call Stack: null, Custom Event ID: -1, Message: Creation of Template clean_ppc_template from VM ppc_vm has been completed. >2014-10-20 15:07:08,066 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-1) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddVmTemplate completed, handling the result. >2014-10-20 15:07:08,066 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-1) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddVmTemplate succeeded, clearing tasks. >2014-10-20 15:07:08,075 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-1) SPMAsyncTask::ClearAsyncTask: Attempting to clear task ba1faaa1-e56a-4aca-8c32-abcc38304491 >2014-10-20 15:07:08,076 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-1) START, SPMClearTaskVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, taskId = ba1faaa1-e56a-4aca-8c32-abcc38304491), log id: 73eb883b >2014-10-20 15:07:08,084 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-1) START, HSMClearTaskVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, taskId=ba1faaa1-e56a-4aca-8c32-abcc38304491), log id: 6c3d14f1 >2014-10-20 15:07:08,121 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-1) FINISH, HSMClearTaskVDSCommand, log id: 6c3d14f1 >2014-10-20 15:07:08,121 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-1) FINISH, SPMClearTaskVDSCommand, log id: 73eb883b >2014-10-20 15:07:08,127 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-1) BaseAsyncTask::removeTaskFromDB: Removed task ba1faaa1-e56a-4aca-8c32-abcc38304491 from DataBase >2014-10-20 15:07:08,127 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-1) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 39ca3782-12c3-42f7-b190-803b1694898d >2014-10-20 15:07:26,230 INFO [org.ovirt.engine.core.bll.AddVmPoolWithVmsCommand] (org.ovirt.thread.pool-7-thread-27) [69d66334] Running command: AddVmPoolWithVmsCommand internal: false. Entities affected : ID: 08ea0169-ae42-4e1b-bb20-839a66cd2010 Type: VdsGroupsAction group CREATE_VM_POOL with role type USER, ID: 30bad43f-c797-4445-b598-a26dbb2a3400 Type: VmTemplateAction group CREATE_VM with role type USER >2014-10-20 15:07:26,381 INFO [org.ovirt.engine.core.bll.AddVmAndAttachToPoolCommand] (org.ovirt.thread.pool-7-thread-27) [2a7ed77d] Running command: AddVmAndAttachToPoolCommand internal: true. Entities affected : ID: 08ea0169-ae42-4e1b-bb20-839a66cd2010 Type: VdsGroups, ID: 30bad43f-c797-4445-b598-a26dbb2a3400 Type: VmTemplate, ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: StorageAction group CREATE_DISK with role type USER >2014-10-20 15:07:26,398 INFO [org.ovirt.engine.core.bll.AddVmCommand] (org.ovirt.thread.pool-7-thread-27) [5f27ea6e] Lock Acquired to object EngineLock [exclusiveLocks= key: ppc_pool-1 value: VM_NAME >, sharedLocks= key: 9a68745c-3578-46d7-ad9c-d9e03c4639dd value: DISK >key: 30bad43f-c797-4445-b598-a26dbb2a3400 value: TEMPLATE >] >2014-10-20 15:07:26,478 INFO [org.ovirt.engine.core.bll.AddVmCommand] (org.ovirt.thread.pool-7-thread-27) [5f27ea6e] Running command: AddVmCommand internal: true. Entities affected : ID: 08ea0169-ae42-4e1b-bb20-839a66cd2010 Type: VdsGroupsAction group CREATE_VM with role type USER, ID: 30bad43f-c797-4445-b598-a26dbb2a3400 Type: VmTemplateAction group CREATE_VM with role type USER, ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: StorageAction group CREATE_DISK with role type USER >2014-10-20 15:07:26,507 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-27) [5f27ea6e] START, SetVmStatusVDSCommand( vmId = 81c4e2f6-2c1e-4684-a991-381976308e16, status = ImageLocked, exit status = Normal), log id: 3497c5f >2014-10-20 15:07:26,514 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-27) [5f27ea6e] FINISH, SetVmStatusVDSCommand, log id: 3497c5f >2014-10-20 15:07:26,820 INFO [org.ovirt.engine.core.bll.CreateSnapshotFromTemplateCommand] (org.ovirt.thread.pool-7-thread-27) [55569f26] Running command: CreateSnapshotFromTemplateCommand internal: true. Entities affected : ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: Storage >2014-10-20 15:07:26,858 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-7-thread-27) [55569f26] START, CreateSnapshotVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = 9a10ab99-58cf-4451-8fc7-ced081f2cfca, imageSizeInBytes = 1073741824, volumeFormat = COW, newImageId = 70f5063d-d6d9-4266-8d69-bd752dc671d7, newImageDescription = , imageId = 4ad0aced-a99f-4f8c-9ac0-a8b16ed9cf95, sourceImageGroupId = 9a68745c-3578-46d7-ad9c-d9e03c4639dd), log id: 431905e3 >2014-10-20 15:07:26,859 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-7-thread-27) [55569f26] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-10-20 15:07:27,006 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-7-thread-27) [55569f26] FINISH, CreateSnapshotVDSCommand, return: 70f5063d-d6d9-4266-8d69-bd752dc671d7, log id: 431905e3 >2014-10-20 15:07:27,009 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-27) [55569f26] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 0b7c6abc-6ec5-4e7f-a34d-f6a1794bf199 >2014-10-20 15:07:27,009 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-7-thread-27) [55569f26] CommandMultiAsyncTasks::AttachTask: Attaching task 8f56fa6c-9265-41c8-94a9-4d8c0006cf21 to command 0b7c6abc-6ec5-4e7f-a34d-f6a1794bf199. >2014-10-20 15:07:27,024 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-7-thread-27) [55569f26] Adding task 8f56fa6c-9265-41c8-94a9-4d8c0006cf21 (Parent Command AddVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-10-20 15:07:27,120 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-27) [55569f26] Correlation ID: 5f27ea6e, Call Stack: null, Custom Event ID: -1, Message: VM ppc_pool-1 creation was initiated by admin. >2014-10-20 15:07:27,121 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-27) [55569f26] BaseAsyncTask::startPollingTask: Starting to poll task 8f56fa6c-9265-41c8-94a9-4d8c0006cf21. >2014-10-20 15:07:27,202 INFO [org.ovirt.engine.core.bll.AddVmToPoolCommand] (org.ovirt.thread.pool-7-thread-27) [d267006] Running command: AddVmToPoolCommand internal: true. Entities affected : ID: 70e155ad-bc0c-4a02-9121-b8995c915ede Type: VmPool >2014-10-20 15:07:27,295 INFO [org.ovirt.engine.core.bll.AddVmAndAttachToPoolCommand] (org.ovirt.thread.pool-7-thread-27) [6f4d7967] Running command: AddVmAndAttachToPoolCommand internal: true. Entities affected : ID: 08ea0169-ae42-4e1b-bb20-839a66cd2010 Type: VdsGroups, ID: 30bad43f-c797-4445-b598-a26dbb2a3400 Type: VmTemplate, ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: StorageAction group CREATE_DISK with role type USER >2014-10-20 15:07:27,310 INFO [org.ovirt.engine.core.bll.AddVmCommand] (org.ovirt.thread.pool-7-thread-27) [75d955cc] Lock Acquired to object EngineLock [exclusiveLocks= key: ppc_pool-2 value: VM_NAME >, sharedLocks= key: 9a68745c-3578-46d7-ad9c-d9e03c4639dd value: DISK >key: 30bad43f-c797-4445-b598-a26dbb2a3400 value: TEMPLATE >] >2014-10-20 15:07:27,358 INFO [org.ovirt.engine.core.bll.AddVmCommand] (org.ovirt.thread.pool-7-thread-27) [75d955cc] Running command: AddVmCommand internal: true. Entities affected : ID: 08ea0169-ae42-4e1b-bb20-839a66cd2010 Type: VdsGroupsAction group CREATE_VM with role type USER, ID: 30bad43f-c797-4445-b598-a26dbb2a3400 Type: VmTemplateAction group CREATE_VM with role type USER, ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: StorageAction group CREATE_DISK with role type USER >2014-10-20 15:07:27,389 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-27) [75d955cc] START, SetVmStatusVDSCommand( vmId = 70893f36-6595-4087-be49-296fb5337807, status = ImageLocked, exit status = Normal), log id: 188f4ff9 >2014-10-20 15:07:27,436 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-27) [75d955cc] FINISH, SetVmStatusVDSCommand, log id: 188f4ff9 >2014-10-20 15:07:27,599 INFO [org.ovirt.engine.core.bll.CreateSnapshotFromTemplateCommand] (org.ovirt.thread.pool-7-thread-27) [7bda4332] Running command: CreateSnapshotFromTemplateCommand internal: true. Entities affected : ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: Storage >2014-10-20 15:07:27,603 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-7-thread-27) [7bda4332] START, CreateSnapshotVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = a8d7b763-63fb-426f-9741-3a3b82daeda6, imageSizeInBytes = 1073741824, volumeFormat = COW, newImageId = c1c93a30-12d8-465b-9a49-9c4157f581cf, newImageDescription = , imageId = 4ad0aced-a99f-4f8c-9ac0-a8b16ed9cf95, sourceImageGroupId = 9a68745c-3578-46d7-ad9c-d9e03c4639dd), log id: 1e382a3b >2014-10-20 15:07:27,603 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-7-thread-27) [7bda4332] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-10-20 15:07:27,834 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-41) Polling and updating Async Tasks: 4 tasks, 1 tasks to poll now >2014-10-20 15:07:27,847 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-7-thread-27) [7bda4332] FINISH, CreateSnapshotVDSCommand, return: c1c93a30-12d8-465b-9a49-9c4157f581cf, log id: 1e382a3b >2014-10-20 15:07:27,849 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-27) [7bda4332] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command aaf7708d-a39b-4f57-aab4-31451d05fc0f >2014-10-20 15:07:27,849 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-7-thread-27) [7bda4332] CommandMultiAsyncTasks::AttachTask: Attaching task b234ff4c-1361-401a-a694-86e85335a02c to command aaf7708d-a39b-4f57-aab4-31451d05fc0f. >2014-10-20 15:07:27,891 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-41) SPMAsyncTask::PollTask: Polling task 8f56fa6c-9265-41c8-94a9-4d8c0006cf21 (Parent Command AddVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-10-20 15:07:27,891 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-41) BaseAsyncTask::onTaskEndSuccess: Task 8f56fa6c-9265-41c8-94a9-4d8c0006cf21 (Parent Command AddVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-10-20 15:07:27,893 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-41) CommandAsyncTask::endActionIfNecessary: All tasks of command 0b7c6abc-6ec5-4e7f-a34d-f6a1794bf199 has ended -> executing endAction >2014-10-20 15:07:27,893 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-41) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: 0b7c6abc-6ec5-4e7f-a34d-f6a1794bf199): calling endAction . >2014-10-20 15:07:27,893 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-7-thread-27) [7bda4332] Adding task b234ff4c-1361-401a-a694-86e85335a02c (Parent Command AddVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-10-20 15:07:27,901 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-31) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction AddVm, executionIndex: 0 >2014-10-20 15:07:28,050 INFO [org.ovirt.engine.core.bll.AddVmCommand] (org.ovirt.thread.pool-7-thread-31) [5f27ea6e] Ending command successfully: org.ovirt.engine.core.bll.AddVmCommand >2014-10-20 15:07:28,148 INFO [org.ovirt.engine.core.bll.CreateSnapshotFromTemplateCommand] (org.ovirt.thread.pool-7-thread-31) [55569f26] Ending command successfully: org.ovirt.engine.core.bll.CreateSnapshotFromTemplateCommand >2014-10-20 15:07:28,152 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-31) [55569f26] START, GetImageInfoVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = 9a10ab99-58cf-4451-8fc7-ced081f2cfca, imageId = 70f5063d-d6d9-4266-8d69-bd752dc671d7), log id: 584a0fef >2014-10-20 15:07:28,197 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-27) [7bda4332] Correlation ID: 75d955cc, Call Stack: null, Custom Event ID: -1, Message: VM ppc_pool-2 creation was initiated by admin. >2014-10-20 15:07:28,197 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-27) [7bda4332] BaseAsyncTask::startPollingTask: Starting to poll task b234ff4c-1361-401a-a694-86e85335a02c. >2014-10-20 15:07:28,219 INFO [org.ovirt.engine.core.bll.AddVmToPoolCommand] (org.ovirt.thread.pool-7-thread-27) [6491ee10] Running command: AddVmToPoolCommand internal: true. Entities affected : ID: 70e155ad-bc0c-4a02-9121-b8995c915ede Type: VmPool >2014-10-20 15:07:28,235 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-27) [6491ee10] Correlation ID: 69d66334, Job ID: 4bb9ce60-e412-4f3d-b623-eb2a1847cc6b, Call Stack: null, Custom Event ID: -1, Message: VM Pool ppc_pool (containing 2 VMs) was created by admin. >2014-10-20 15:07:28,243 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-31) [55569f26] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@5f4801ce, log id: 584a0fef >2014-10-20 15:07:28,274 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-31) START, SetVmStatusVDSCommand( vmId = 81c4e2f6-2c1e-4684-a991-381976308e16, status = Down, exit status = Normal), log id: 381cc230 >2014-10-20 15:07:28,290 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-31) FINISH, SetVmStatusVDSCommand, log id: 381cc230 >2014-10-20 15:07:28,292 INFO [org.ovirt.engine.core.bll.AddVmCommand] (org.ovirt.thread.pool-7-thread-31) Lock freed to object EngineLock [exclusiveLocks= key: ppc_pool-1 value: VM_NAME >, sharedLocks= key: 9a68745c-3578-46d7-ad9c-d9e03c4639dd value: DISK >key: 30bad43f-c797-4445-b598-a26dbb2a3400 value: TEMPLATE >] >2014-10-20 15:07:28,300 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-31) Correlation ID: 5f27ea6e, Call Stack: null, Custom Event ID: -1, Message: VM ppc_pool-1 creation has been completed. >2014-10-20 15:07:28,300 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-31) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddVm completed, handling the result. >2014-10-20 15:07:28,300 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-31) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddVm succeeded, clearing tasks. >2014-10-20 15:07:28,300 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-31) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 8f56fa6c-9265-41c8-94a9-4d8c0006cf21 >2014-10-20 15:07:28,300 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-31) START, SPMClearTaskVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, taskId = 8f56fa6c-9265-41c8-94a9-4d8c0006cf21), log id: 6eb559ce >2014-10-20 15:07:28,302 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-31) START, HSMClearTaskVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, taskId=8f56fa6c-9265-41c8-94a9-4d8c0006cf21), log id: 441b1c2a >2014-10-20 15:07:28,387 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-31) FINISH, HSMClearTaskVDSCommand, log id: 441b1c2a >2014-10-20 15:07:28,387 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-31) FINISH, SPMClearTaskVDSCommand, log id: 6eb559ce >2014-10-20 15:07:28,392 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-31) BaseAsyncTask::removeTaskFromDB: Removed task 8f56fa6c-9265-41c8-94a9-4d8c0006cf21 from DataBase >2014-10-20 15:07:28,393 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-31) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 0b7c6abc-6ec5-4e7f-a34d-f6a1794bf199 >2014-10-20 15:07:37,446 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-73) Setting new tasks map. The map contains now 3 tasks >2014-10-20 15:07:37,894 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-76) Polling and updating Async Tasks: 3 tasks, 1 tasks to poll now >2014-10-20 15:07:37,906 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-76) SPMAsyncTask::PollTask: Polling task b234ff4c-1361-401a-a694-86e85335a02c (Parent Command AddVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-10-20 15:07:37,906 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-76) BaseAsyncTask::onTaskEndSuccess: Task b234ff4c-1361-401a-a694-86e85335a02c (Parent Command AddVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-10-20 15:07:37,907 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-76) CommandAsyncTask::endActionIfNecessary: All tasks of command aaf7708d-a39b-4f57-aab4-31451d05fc0f has ended -> executing endAction >2014-10-20 15:07:37,907 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-76) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: aaf7708d-a39b-4f57-aab4-31451d05fc0f): calling endAction . >2014-10-20 15:07:37,908 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-46) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction AddVm, executionIndex: 0 >2014-10-20 15:07:37,929 INFO [org.ovirt.engine.core.bll.AddVmCommand] (org.ovirt.thread.pool-7-thread-46) [75d955cc] Ending command successfully: org.ovirt.engine.core.bll.AddVmCommand >2014-10-20 15:07:37,940 INFO [org.ovirt.engine.core.bll.CreateSnapshotFromTemplateCommand] (org.ovirt.thread.pool-7-thread-46) [7bda4332] Ending command successfully: org.ovirt.engine.core.bll.CreateSnapshotFromTemplateCommand >2014-10-20 15:07:37,944 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-46) [7bda4332] START, GetImageInfoVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = a8d7b763-63fb-426f-9741-3a3b82daeda6, imageId = c1c93a30-12d8-465b-9a49-9c4157f581cf), log id: 5aaae2ef >2014-10-20 15:07:37,986 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-46) [7bda4332] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@b867f3c3, log id: 5aaae2ef >2014-10-20 15:07:37,995 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-46) START, SetVmStatusVDSCommand( vmId = 70893f36-6595-4087-be49-296fb5337807, status = Down, exit status = Normal), log id: 529018e9 >2014-10-20 15:07:38,001 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-46) FINISH, SetVmStatusVDSCommand, log id: 529018e9 >2014-10-20 15:07:38,002 INFO [org.ovirt.engine.core.bll.AddVmCommand] (org.ovirt.thread.pool-7-thread-46) Lock freed to object EngineLock [exclusiveLocks= key: ppc_pool-2 value: VM_NAME >, sharedLocks= key: 9a68745c-3578-46d7-ad9c-d9e03c4639dd value: DISK >key: 30bad43f-c797-4445-b598-a26dbb2a3400 value: TEMPLATE >] >2014-10-20 15:07:38,010 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-46) Correlation ID: 75d955cc, Call Stack: null, Custom Event ID: -1, Message: VM ppc_pool-2 creation has been completed. >2014-10-20 15:07:38,010 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-46) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddVm completed, handling the result. >2014-10-20 15:07:38,010 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-46) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddVm succeeded, clearing tasks. >2014-10-20 15:07:38,010 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-46) SPMAsyncTask::ClearAsyncTask: Attempting to clear task b234ff4c-1361-401a-a694-86e85335a02c >2014-10-20 15:07:38,010 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-46) START, SPMClearTaskVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, taskId = b234ff4c-1361-401a-a694-86e85335a02c), log id: 54ac99c7 >2014-10-20 15:07:38,012 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-46) START, HSMClearTaskVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, taskId=b234ff4c-1361-401a-a694-86e85335a02c), log id: fa4033f >2014-10-20 15:07:38,055 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-46) FINISH, HSMClearTaskVDSCommand, log id: fa4033f >2014-10-20 15:07:38,055 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-46) FINISH, SPMClearTaskVDSCommand, log id: 54ac99c7 >2014-10-20 15:07:38,064 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-46) BaseAsyncTask::removeTaskFromDB: Removed task b234ff4c-1361-401a-a694-86e85335a02c from DataBase >2014-10-20 15:07:38,064 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-46) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity aaf7708d-a39b-4f57-aab4-31451d05fc0f >2014-10-20 15:07:39,553 INFO [org.ovirt.engine.core.bll.RemoveVmFromPoolCommand] (org.ovirt.thread.pool-7-thread-2) [10d9591c] Running command: RemoveVmFromPoolCommand internal: false. Entities affected : ID: 70e155ad-bc0c-4a02-9121-b8995c915ede Type: VmPoolAction group EDIT_VM_POOL_CONFIGURATION with role type USER >2014-10-20 15:07:39,573 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-2) [10d9591c] Correlation ID: 10d9591c, Call Stack: null, Custom Event ID: -1, Message: VM ppc_pool-1 was removed from VM Pool ppc_pool by admin. >2014-10-20 15:07:48,573 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (ajp-/127.0.0.1:8702-8) [103f78c8] Lock Acquired to object EngineLock [exclusiveLocks= key: 81c4e2f6-2c1e-4684-a991-381976308e16 value: VM >, sharedLocks= ] >2014-10-20 15:07:48,622 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-14) [103f78c8] Running command: RemoveVmCommand internal: false. Entities affected : ID: 81c4e2f6-2c1e-4684-a991-381976308e16 Type: VMAction group DELETE_VM with role type USER >2014-10-20 15:07:48,624 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-14) [103f78c8] START, SetVmStatusVDSCommand( vmId = 81c4e2f6-2c1e-4684-a991-381976308e16, status = ImageLocked, exit status = Normal), log id: 36a66df5 >2014-10-20 15:07:48,639 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-14) [103f78c8] FINISH, SetVmStatusVDSCommand, log id: 36a66df5 >2014-10-20 15:07:48,651 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-14) [103f78c8] Lock freed to object EngineLock [exclusiveLocks= key: 81c4e2f6-2c1e-4684-a991-381976308e16 value: VM >, sharedLocks= ] >2014-10-20 15:07:48,687 INFO [org.ovirt.engine.core.bll.RemoveAllVmImagesCommand] (org.ovirt.thread.pool-7-thread-14) [3dd9f973] Running command: RemoveAllVmImagesCommand internal: true. Entities affected : ID: 81c4e2f6-2c1e-4684-a991-381976308e16 Type: VM >2014-10-20 15:07:48,696 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-7-thread-14) [77e6ab7f] Running command: RemoveImageCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage >2014-10-20 15:07:48,800 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-7-thread-14) [77e6ab7f] START, DeleteImageGroupVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = 9a10ab99-58cf-4451-8fc7-ced081f2cfca, postZeros = false, forceDelete = false), log id: 65ca4bf3 >2014-10-20 15:07:48,969 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-7-thread-14) [77e6ab7f] FINISH, DeleteImageGroupVDSCommand, log id: 65ca4bf3 >2014-10-20 15:07:48,973 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-14) [77e6ab7f] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 545e3274-098e-498d-9da8-d287725fe750 >2014-10-20 15:07:48,973 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-7-thread-14) [77e6ab7f] CommandMultiAsyncTasks::AttachTask: Attaching task 4f277b51-c509-4c72-823e-d898ada9a32b to command 545e3274-098e-498d-9da8-d287725fe750. >2014-10-20 15:07:49,003 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-7-thread-14) [77e6ab7f] Adding task 4f277b51-c509-4c72-823e-d898ada9a32b (Parent Command RemoveVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-10-20 15:07:49,148 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-14) [77e6ab7f] Correlation ID: 103f78c8, Job ID: 5bae765e-2ae2-47a1-ba25-b364885d5e55, Call Stack: null, Custom Event ID: -1, Message: VM ppc_pool-1 was successfully removed. >2014-10-20 15:07:49,148 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-14) [77e6ab7f] BaseAsyncTask::startPollingTask: Starting to poll task 4f277b51-c509-4c72-823e-d898ada9a32b. >2014-10-20 15:07:56,045 INFO [org.ovirt.engine.core.bll.RemoveVmFromPoolCommand] (org.ovirt.thread.pool-7-thread-25) [5e29156e] Running command: RemoveVmFromPoolCommand internal: false. Entities affected : ID: 70e155ad-bc0c-4a02-9121-b8995c915ede Type: VmPoolAction group EDIT_VM_POOL_CONFIGURATION with role type USER >2014-10-20 15:07:56,054 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-25) [5e29156e] Correlation ID: 5e29156e, Call Stack: null, Custom Event ID: -1, Message: VM ppc_pool-2 was removed from VM Pool ppc_pool by admin. >2014-10-20 15:07:56,061 INFO [org.ovirt.engine.core.bll.RemoveVmPoolCommand] (org.ovirt.thread.pool-7-thread-25) [33a563f] Running command: RemoveVmPoolCommand internal: true. Entities affected : ID: 70e155ad-bc0c-4a02-9121-b8995c915ede Type: VmPoolAction group DELETE_VM_POOL with role type USER >2014-10-20 15:07:56,070 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-25) [33a563f] Correlation ID: 33a563f, Call Stack: null, Custom Event ID: -1, Message: VM Pool ppc_pool was removed by admin. >2014-10-20 15:07:57,908 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-51) Polling and updating Async Tasks: 4 tasks, 1 tasks to poll now >2014-10-20 15:07:57,918 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-51) SPMAsyncTask::PollTask: Polling task 4f277b51-c509-4c72-823e-d898ada9a32b (Parent Command RemoveVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-10-20 15:07:57,922 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-51) BaseAsyncTask::onTaskEndSuccess: Task 4f277b51-c509-4c72-823e-d898ada9a32b (Parent Command RemoveVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-10-20 15:07:57,924 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-51) CommandAsyncTask::endActionIfNecessary: All tasks of command 545e3274-098e-498d-9da8-d287725fe750 has ended -> executing endAction >2014-10-20 15:07:57,924 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-51) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: 545e3274-098e-498d-9da8-d287725fe750): calling endAction . >2014-10-20 15:07:57,924 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-28) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction RemoveVm, executionIndex: 0 >2014-10-20 15:07:57,944 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-28) Ending command successfully: org.ovirt.engine.core.bll.RemoveVmCommand >2014-10-20 15:07:57,946 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-28) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RemoveVm completed, handling the result. >2014-10-20 15:07:57,946 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-28) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RemoveVm succeeded, clearing tasks. >2014-10-20 15:07:57,948 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-28) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 4f277b51-c509-4c72-823e-d898ada9a32b >2014-10-20 15:07:57,949 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-28) START, SPMClearTaskVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, taskId = 4f277b51-c509-4c72-823e-d898ada9a32b), log id: 4d159d87 >2014-10-20 15:07:57,950 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-28) START, HSMClearTaskVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, taskId=4f277b51-c509-4c72-823e-d898ada9a32b), log id: 4619319a >2014-10-20 15:07:57,979 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-28) FINISH, HSMClearTaskVDSCommand, log id: 4619319a >2014-10-20 15:07:57,979 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-28) FINISH, SPMClearTaskVDSCommand, log id: 4d159d87 >2014-10-20 15:07:57,984 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-28) BaseAsyncTask::removeTaskFromDB: Removed task 4f277b51-c509-4c72-823e-d898ada9a32b from DataBase >2014-10-20 15:07:57,984 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-28) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 545e3274-098e-498d-9da8-d287725fe750 >2014-10-20 15:08:01,582 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (ajp-/127.0.0.1:8702-8) [65792329] Lock Acquired to object EngineLock [exclusiveLocks= key: 70893f36-6595-4087-be49-296fb5337807 value: VM >, sharedLocks= ] >2014-10-20 15:08:01,713 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-33) [65792329] Running command: RemoveVmCommand internal: false. Entities affected : ID: 70893f36-6595-4087-be49-296fb5337807 Type: VMAction group DELETE_VM with role type USER >2014-10-20 15:08:01,714 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-33) [65792329] START, SetVmStatusVDSCommand( vmId = 70893f36-6595-4087-be49-296fb5337807, status = ImageLocked, exit status = Normal), log id: cef66e3 >2014-10-20 15:08:01,718 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-33) [65792329] FINISH, SetVmStatusVDSCommand, log id: cef66e3 >2014-10-20 15:08:01,724 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-33) [65792329] Lock freed to object EngineLock [exclusiveLocks= key: 70893f36-6595-4087-be49-296fb5337807 value: VM >, sharedLocks= ] >2014-10-20 15:08:01,810 INFO [org.ovirt.engine.core.bll.RemoveAllVmImagesCommand] (org.ovirt.thread.pool-7-thread-33) [1858a4a3] Running command: RemoveAllVmImagesCommand internal: true. Entities affected : ID: 70893f36-6595-4087-be49-296fb5337807 Type: VM >2014-10-20 15:08:01,814 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-7-thread-33) [4fae1e7f] Running command: RemoveImageCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage >2014-10-20 15:08:01,871 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-7-thread-33) [4fae1e7f] START, DeleteImageGroupVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = a8d7b763-63fb-426f-9741-3a3b82daeda6, postZeros = false, forceDelete = false), log id: 3213d734 >2014-10-20 15:08:02,040 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-7-thread-33) [4fae1e7f] FINISH, DeleteImageGroupVDSCommand, log id: 3213d734 >2014-10-20 15:08:02,044 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-33) [4fae1e7f] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 749a8249-4062-4a10-9b64-53c28d1a58a0 >2014-10-20 15:08:02,044 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-7-thread-33) [4fae1e7f] CommandMultiAsyncTasks::AttachTask: Attaching task bfc2b8f0-7280-4594-8ee9-e2029188da0f to command 749a8249-4062-4a10-9b64-53c28d1a58a0. >2014-10-20 15:08:02,054 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-7-thread-33) [4fae1e7f] Adding task bfc2b8f0-7280-4594-8ee9-e2029188da0f (Parent Command RemoveVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-10-20 15:08:02,184 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-33) [4fae1e7f] Correlation ID: 65792329, Job ID: d9ead543-d6e5-4cea-9ab0-cc6e38d741bc, Call Stack: null, Custom Event ID: -1, Message: VM ppc_pool-2 was successfully removed. >2014-10-20 15:08:02,185 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-33) [4fae1e7f] BaseAsyncTask::startPollingTask: Starting to poll task bfc2b8f0-7280-4594-8ee9-e2029188da0f. >2014-10-20 15:08:07,925 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-88) Polling and updating Async Tasks: 5 tasks, 1 tasks to poll now >2014-10-20 15:08:07,936 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-88) SPMAsyncTask::PollTask: Polling task bfc2b8f0-7280-4594-8ee9-e2029188da0f (Parent Command RemoveVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-10-20 15:08:07,941 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-88) BaseAsyncTask::onTaskEndSuccess: Task bfc2b8f0-7280-4594-8ee9-e2029188da0f (Parent Command RemoveVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-10-20 15:08:07,942 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-88) CommandAsyncTask::endActionIfNecessary: All tasks of command 749a8249-4062-4a10-9b64-53c28d1a58a0 has ended -> executing endAction >2014-10-20 15:08:07,942 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-88) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: 749a8249-4062-4a10-9b64-53c28d1a58a0): calling endAction . >2014-10-20 15:08:07,943 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-44) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction RemoveVm, executionIndex: 0 >2014-10-20 15:08:08,016 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-44) Ending command successfully: org.ovirt.engine.core.bll.RemoveVmCommand >2014-10-20 15:08:08,018 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-44) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RemoveVm completed, handling the result. >2014-10-20 15:08:08,018 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-44) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RemoveVm succeeded, clearing tasks. >2014-10-20 15:08:08,028 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-44) SPMAsyncTask::ClearAsyncTask: Attempting to clear task bfc2b8f0-7280-4594-8ee9-e2029188da0f >2014-10-20 15:08:08,028 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-44) START, SPMClearTaskVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, taskId = bfc2b8f0-7280-4594-8ee9-e2029188da0f), log id: 588c4102 >2014-10-20 15:08:08,030 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-44) START, HSMClearTaskVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, taskId=bfc2b8f0-7280-4594-8ee9-e2029188da0f), log id: 600017b8 >2014-10-20 15:08:08,057 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-44) FINISH, HSMClearTaskVDSCommand, log id: 600017b8 >2014-10-20 15:08:08,058 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-44) FINISH, SPMClearTaskVDSCommand, log id: 588c4102 >2014-10-20 15:08:08,061 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-44) BaseAsyncTask::removeTaskFromDB: Removed task bfc2b8f0-7280-4594-8ee9-e2029188da0f from DataBase >2014-10-20 15:08:08,062 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-44) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 749a8249-4062-4a10-9b64-53c28d1a58a0 >2014-10-20 15:08:18,658 INFO [org.ovirt.engine.core.bll.AddVmPoolWithVmsCommand] (org.ovirt.thread.pool-7-thread-9) [d25cc0a] Running command: AddVmPoolWithVmsCommand internal: false. Entities affected : ID: 63702093-c04f-417c-8045-aa56334b54ee Type: VdsGroupsAction group CREATE_VM_POOL with role type USER, ID: b76c08ce-2fdd-4047-93bb-326d95d5cd1d Type: VmTemplateAction group CREATE_VM with role type USER >2014-10-20 15:08:18,874 INFO [org.ovirt.engine.core.bll.AddVmAndAttachToPoolCommand] (org.ovirt.thread.pool-7-thread-9) [49a4853e] Running command: AddVmAndAttachToPoolCommand internal: true. Entities affected : ID: 63702093-c04f-417c-8045-aa56334b54ee Type: VdsGroups, ID: b76c08ce-2fdd-4047-93bb-326d95d5cd1d Type: VmTemplate, ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: StorageAction group CREATE_DISK with role type USER >2014-10-20 15:08:18,915 INFO [org.ovirt.engine.core.bll.AddVmCommand] (org.ovirt.thread.pool-7-thread-9) [4fcd6e29] Lock Acquired to object EngineLock [exclusiveLocks= key: ppc_pool-1 value: VM_NAME >, sharedLocks= key: b76c08ce-2fdd-4047-93bb-326d95d5cd1d value: TEMPLATE >key: 3778d539-c7de-48fc-be30-9a02bf87665a value: DISK >] >2014-10-20 15:08:18,971 INFO [org.ovirt.engine.core.bll.AddVmCommand] (org.ovirt.thread.pool-7-thread-9) [4fcd6e29] Running command: AddVmCommand internal: true. Entities affected : ID: 63702093-c04f-417c-8045-aa56334b54ee Type: VdsGroupsAction group CREATE_VM with role type USER, ID: b76c08ce-2fdd-4047-93bb-326d95d5cd1d Type: VmTemplateAction group CREATE_VM with role type USER, ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: StorageAction group CREATE_DISK with role type USER >2014-10-20 15:08:18,993 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-9) [4fcd6e29] START, SetVmStatusVDSCommand( vmId = 42998ef0-6fc3-4e28-bf56-b388913b547d, status = ImageLocked, exit status = Normal), log id: 2e9217c3 >2014-10-20 15:08:19,000 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-9) [4fcd6e29] FINISH, SetVmStatusVDSCommand, log id: 2e9217c3 >2014-10-20 15:08:19,046 INFO [org.ovirt.engine.core.bll.CreateSnapshotFromTemplateCommand] (org.ovirt.thread.pool-7-thread-9) [441a8d36] Running command: CreateSnapshotFromTemplateCommand internal: true. Entities affected : ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: Storage >2014-10-20 15:08:19,050 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-7-thread-9) [441a8d36] START, CreateSnapshotVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = 075b5199-62ff-44cd-8b8e-e1eccedfcb6c, imageSizeInBytes = 1073741824, volumeFormat = COW, newImageId = 871887a7-e181-41e7-9c97-e1909cc2d8c5, newImageDescription = , imageId = a9c8a68a-7a40-4b35-95e8-b8f8c8d67057, sourceImageGroupId = 3778d539-c7de-48fc-be30-9a02bf87665a), log id: 699b6d05 >2014-10-20 15:08:19,059 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-7-thread-9) [441a8d36] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-10-20 15:08:19,203 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-7-thread-9) [441a8d36] FINISH, CreateSnapshotVDSCommand, return: 871887a7-e181-41e7-9c97-e1909cc2d8c5, log id: 699b6d05 >2014-10-20 15:08:19,204 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-9) [441a8d36] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 05c5d590-b24f-4935-986f-de6ee2d3a1d3 >2014-10-20 15:08:19,204 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-7-thread-9) [441a8d36] CommandMultiAsyncTasks::AttachTask: Attaching task 79d7ea57-5bc2-4195-86f5-e93b0b29cf8c to command 05c5d590-b24f-4935-986f-de6ee2d3a1d3. >2014-10-20 15:08:19,348 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-7-thread-9) [441a8d36] Adding task 79d7ea57-5bc2-4195-86f5-e93b0b29cf8c (Parent Command AddVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-10-20 15:08:19,453 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-9) [441a8d36] Correlation ID: 4fcd6e29, Call Stack: null, Custom Event ID: -1, Message: VM ppc_pool-1 creation was initiated by admin. >2014-10-20 15:08:19,453 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-9) [441a8d36] BaseAsyncTask::startPollingTask: Starting to poll task 79d7ea57-5bc2-4195-86f5-e93b0b29cf8c. >2014-10-20 15:08:19,473 INFO [org.ovirt.engine.core.bll.AddVmToPoolCommand] (org.ovirt.thread.pool-7-thread-9) [7a63f7cc] Running command: AddVmToPoolCommand internal: true. Entities affected : ID: 8f1febe6-db76-4ff6-ab30-7f42cb8cf781 Type: VmPool >2014-10-20 15:08:19,542 INFO [org.ovirt.engine.core.bll.AddVmAndAttachToPoolCommand] (org.ovirt.thread.pool-7-thread-9) [34845d9c] Running command: AddVmAndAttachToPoolCommand internal: true. Entities affected : ID: 63702093-c04f-417c-8045-aa56334b54ee Type: VdsGroups, ID: b76c08ce-2fdd-4047-93bb-326d95d5cd1d Type: VmTemplate, ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: StorageAction group CREATE_DISK with role type USER >2014-10-20 15:08:19,556 INFO [org.ovirt.engine.core.bll.AddVmCommand] (org.ovirt.thread.pool-7-thread-9) [69edbb4c] Lock Acquired to object EngineLock [exclusiveLocks= key: ppc_pool-2 value: VM_NAME >, sharedLocks= key: b76c08ce-2fdd-4047-93bb-326d95d5cd1d value: TEMPLATE >key: 3778d539-c7de-48fc-be30-9a02bf87665a value: DISK >] >2014-10-20 15:08:19,605 INFO [org.ovirt.engine.core.bll.AddVmCommand] (org.ovirt.thread.pool-7-thread-9) [69edbb4c] Running command: AddVmCommand internal: true. Entities affected : ID: 63702093-c04f-417c-8045-aa56334b54ee Type: VdsGroupsAction group CREATE_VM with role type USER, ID: b76c08ce-2fdd-4047-93bb-326d95d5cd1d Type: VmTemplateAction group CREATE_VM with role type USER, ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: StorageAction group CREATE_DISK with role type USER >2014-10-20 15:08:19,635 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-9) [69edbb4c] START, SetVmStatusVDSCommand( vmId = e1c392c2-1ccc-433a-9f30-ea302d6ece77, status = ImageLocked, exit status = Normal), log id: 7cacf66e >2014-10-20 15:08:19,651 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-9) [69edbb4c] FINISH, SetVmStatusVDSCommand, log id: 7cacf66e >2014-10-20 15:08:20,017 INFO [org.ovirt.engine.core.bll.CreateSnapshotFromTemplateCommand] (org.ovirt.thread.pool-7-thread-9) [6446ecc6] Running command: CreateSnapshotFromTemplateCommand internal: true. Entities affected : ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: Storage >2014-10-20 15:08:20,021 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-7-thread-9) [6446ecc6] START, CreateSnapshotVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = ff51fcdc-f313-49ef-a677-dfa8cbc00820, imageSizeInBytes = 1073741824, volumeFormat = COW, newImageId = cedeff95-ffb9-45ed-9fb4-816781359807, newImageDescription = , imageId = a9c8a68a-7a40-4b35-95e8-b8f8c8d67057, sourceImageGroupId = 3778d539-c7de-48fc-be30-9a02bf87665a), log id: 6489b478 >2014-10-20 15:08:20,022 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-7-thread-9) [6446ecc6] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-10-20 15:08:20,422 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-7-thread-9) [6446ecc6] FINISH, CreateSnapshotVDSCommand, return: cedeff95-ffb9-45ed-9fb4-816781359807, log id: 6489b478 >2014-10-20 15:08:20,424 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-9) [6446ecc6] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 2a17bcba-63b6-4a69-b31b-7102aec3d5a8 >2014-10-20 15:08:20,424 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-7-thread-9) [6446ecc6] CommandMultiAsyncTasks::AttachTask: Attaching task c9323a30-d7d7-4a18-87d0-231dac1696af to command 2a17bcba-63b6-4a69-b31b-7102aec3d5a8. >2014-10-20 15:08:20,492 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-7-thread-9) [6446ecc6] Adding task c9323a30-d7d7-4a18-87d0-231dac1696af (Parent Command AddVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-10-20 15:08:20,579 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-9) [6446ecc6] Correlation ID: 69edbb4c, Call Stack: null, Custom Event ID: -1, Message: VM ppc_pool-2 creation was initiated by admin. >2014-10-20 15:08:20,579 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-9) [6446ecc6] BaseAsyncTask::startPollingTask: Starting to poll task c9323a30-d7d7-4a18-87d0-231dac1696af. >2014-10-20 15:08:20,591 INFO [org.ovirt.engine.core.bll.AddVmToPoolCommand] (org.ovirt.thread.pool-7-thread-9) [7f722681] Running command: AddVmToPoolCommand internal: true. Entities affected : ID: 8f1febe6-db76-4ff6-ab30-7f42cb8cf781 Type: VmPool >2014-10-20 15:08:20,619 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-9) [7f722681] Correlation ID: d25cc0a, Job ID: a39fb633-0577-47f9-99b7-1d2ecec43ec7, Call Stack: null, Custom Event ID: -1, Message: VM Pool ppc_pool (containing 2 VMs) was created by admin. >2014-10-20 15:08:27,944 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-65) Polling and updating Async Tasks: 7 tasks, 2 tasks to poll now >2014-10-20 15:08:27,957 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-65) SPMAsyncTask::PollTask: Polling task 79d7ea57-5bc2-4195-86f5-e93b0b29cf8c (Parent Command AddVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-10-20 15:08:27,957 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-65) BaseAsyncTask::onTaskEndSuccess: Task 79d7ea57-5bc2-4195-86f5-e93b0b29cf8c (Parent Command AddVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-10-20 15:08:27,958 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-65) CommandAsyncTask::endActionIfNecessary: All tasks of command 05c5d590-b24f-4935-986f-de6ee2d3a1d3 has ended -> executing endAction >2014-10-20 15:08:27,958 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-65) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: 05c5d590-b24f-4935-986f-de6ee2d3a1d3): calling endAction . >2014-10-20 15:08:27,959 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-65) SPMAsyncTask::PollTask: Polling task c9323a30-d7d7-4a18-87d0-231dac1696af (Parent Command AddVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-10-20 15:08:27,959 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-65) BaseAsyncTask::onTaskEndSuccess: Task c9323a30-d7d7-4a18-87d0-231dac1696af (Parent Command AddVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-10-20 15:08:27,959 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-23) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction AddVm, executionIndex: 0 >2014-10-20 15:08:27,960 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-65) CommandAsyncTask::endActionIfNecessary: All tasks of command 2a17bcba-63b6-4a69-b31b-7102aec3d5a8 has ended -> executing endAction >2014-10-20 15:08:27,960 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-65) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: 2a17bcba-63b6-4a69-b31b-7102aec3d5a8): calling endAction . >2014-10-20 15:08:27,960 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-25) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction AddVm, executionIndex: 0 >2014-10-20 15:08:28,016 INFO [org.ovirt.engine.core.bll.AddVmCommand] (org.ovirt.thread.pool-7-thread-23) [4fcd6e29] Ending command successfully: org.ovirt.engine.core.bll.AddVmCommand >2014-10-20 15:08:28,033 INFO [org.ovirt.engine.core.bll.AddVmCommand] (org.ovirt.thread.pool-7-thread-25) [69edbb4c] Ending command successfully: org.ovirt.engine.core.bll.AddVmCommand >2014-10-20 15:08:28,039 INFO [org.ovirt.engine.core.bll.CreateSnapshotFromTemplateCommand] (org.ovirt.thread.pool-7-thread-23) [441a8d36] Ending command successfully: org.ovirt.engine.core.bll.CreateSnapshotFromTemplateCommand >2014-10-20 15:08:28,052 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-23) [441a8d36] START, GetImageInfoVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = 075b5199-62ff-44cd-8b8e-e1eccedfcb6c, imageId = 871887a7-e181-41e7-9c97-e1909cc2d8c5), log id: 29b4afb >2014-10-20 15:08:28,071 INFO [org.ovirt.engine.core.bll.CreateSnapshotFromTemplateCommand] (org.ovirt.thread.pool-7-thread-25) [6446ecc6] Ending command successfully: org.ovirt.engine.core.bll.CreateSnapshotFromTemplateCommand >2014-10-20 15:08:28,085 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-25) [6446ecc6] START, GetImageInfoVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = ff51fcdc-f313-49ef-a677-dfa8cbc00820, imageId = cedeff95-ffb9-45ed-9fb4-816781359807), log id: 3e4b7588 >2014-10-20 15:08:28,091 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-23) [441a8d36] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@accbc83b, log id: 29b4afb >2014-10-20 15:08:28,098 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-23) START, SetVmStatusVDSCommand( vmId = 42998ef0-6fc3-4e28-bf56-b388913b547d, status = Down, exit status = Normal), log id: 3b01f7d2 >2014-10-20 15:08:28,102 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-23) FINISH, SetVmStatusVDSCommand, log id: 3b01f7d2 >2014-10-20 15:08:28,103 INFO [org.ovirt.engine.core.bll.AddVmCommand] (org.ovirt.thread.pool-7-thread-23) Lock freed to object EngineLock [exclusiveLocks= key: ppc_pool-1 value: VM_NAME >, sharedLocks= key: b76c08ce-2fdd-4047-93bb-326d95d5cd1d value: TEMPLATE >key: 3778d539-c7de-48fc-be30-9a02bf87665a value: DISK >] >2014-10-20 15:08:28,108 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-23) Correlation ID: 4fcd6e29, Call Stack: null, Custom Event ID: -1, Message: VM ppc_pool-1 creation has been completed. >2014-10-20 15:08:28,108 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-23) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddVm completed, handling the result. >2014-10-20 15:08:28,109 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-23) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddVm succeeded, clearing tasks. >2014-10-20 15:08:28,109 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-23) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 79d7ea57-5bc2-4195-86f5-e93b0b29cf8c >2014-10-20 15:08:28,109 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-23) START, SPMClearTaskVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, taskId = 79d7ea57-5bc2-4195-86f5-e93b0b29cf8c), log id: 6ed05fb5 >2014-10-20 15:08:28,132 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-25) [6446ecc6] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@96239111, log id: 3e4b7588 >2014-10-20 15:08:28,136 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-23) START, HSMClearTaskVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, taskId=79d7ea57-5bc2-4195-86f5-e93b0b29cf8c), log id: 60d9328a >2014-10-20 15:08:28,139 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-25) START, SetVmStatusVDSCommand( vmId = e1c392c2-1ccc-433a-9f30-ea302d6ece77, status = Down, exit status = Normal), log id: 4fe77e2b >2014-10-20 15:08:28,143 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-25) FINISH, SetVmStatusVDSCommand, log id: 4fe77e2b >2014-10-20 15:08:28,145 INFO [org.ovirt.engine.core.bll.AddVmCommand] (org.ovirt.thread.pool-7-thread-25) Lock freed to object EngineLock [exclusiveLocks= key: ppc_pool-2 value: VM_NAME >, sharedLocks= key: b76c08ce-2fdd-4047-93bb-326d95d5cd1d value: TEMPLATE >key: 3778d539-c7de-48fc-be30-9a02bf87665a value: DISK >] >2014-10-20 15:08:28,150 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-25) Correlation ID: 69edbb4c, Call Stack: null, Custom Event ID: -1, Message: VM ppc_pool-2 creation has been completed. >2014-10-20 15:08:28,150 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-25) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddVm completed, handling the result. >2014-10-20 15:08:28,150 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-25) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddVm succeeded, clearing tasks. >2014-10-20 15:08:28,150 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-25) SPMAsyncTask::ClearAsyncTask: Attempting to clear task c9323a30-d7d7-4a18-87d0-231dac1696af >2014-10-20 15:08:28,150 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-25) START, SPMClearTaskVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, taskId = c9323a30-d7d7-4a18-87d0-231dac1696af), log id: 5c675953 >2014-10-20 15:08:28,181 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-23) FINISH, HSMClearTaskVDSCommand, log id: 60d9328a >2014-10-20 15:08:28,181 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-23) FINISH, SPMClearTaskVDSCommand, log id: 6ed05fb5 >2014-10-20 15:08:28,183 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-25) START, HSMClearTaskVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, taskId=c9323a30-d7d7-4a18-87d0-231dac1696af), log id: 519878a >2014-10-20 15:08:28,186 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-23) BaseAsyncTask::removeTaskFromDB: Removed task 79d7ea57-5bc2-4195-86f5-e93b0b29cf8c from DataBase >2014-10-20 15:08:28,186 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-23) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 05c5d590-b24f-4935-986f-de6ee2d3a1d3 >2014-10-20 15:08:28,234 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-25) FINISH, HSMClearTaskVDSCommand, log id: 519878a >2014-10-20 15:08:28,235 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-25) FINISH, SPMClearTaskVDSCommand, log id: 5c675953 >2014-10-20 15:08:28,240 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-25) BaseAsyncTask::removeTaskFromDB: Removed task c9323a30-d7d7-4a18-87d0-231dac1696af from DataBase >2014-10-20 15:08:28,240 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-25) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 2a17bcba-63b6-4a69-b31b-7102aec3d5a8 >2014-10-20 15:08:34,291 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-10) Running command: LoginUserCommand internal: false. >2014-10-20 15:08:34,294 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-10-20 15:08:34,308 INFO [org.ovirt.engine.core.bll.aaa.LogoutBySessionCommand] (ajp-/127.0.0.1:8702-10) [3bfe4643] Running command: LogoutBySessionCommand internal: false. >2014-10-20 15:08:34,311 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-10) [1da5f8b5] Running command: LogoutUserCommand internal: false. >2014-10-20 15:08:34,314 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) [1da5f8b5] Correlation ID: 1da5f8b5, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-10-20 15:08:37,447 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-99) Setting new tasks map. The map contains now 5 tasks >2014-10-20 15:08:41,479 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-7) Running command: LoginUserCommand internal: false. >2014-10-20 15:08:41,484 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-10-20 15:08:42,108 INFO [org.ovirt.engine.core.bll.aaa.LogoutBySessionCommand] (ajp-/127.0.0.1:8702-7) [3b5c7931] Running command: LogoutBySessionCommand internal: false. >2014-10-20 15:08:42,151 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-7) [2d24511e] Running command: LogoutUserCommand internal: false. >2014-10-20 15:08:42,154 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [2d24511e] Correlation ID: 2d24511e, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-10-20 15:08:48,357 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-3) Running command: LoginUserCommand internal: false. >2014-10-20 15:08:48,368 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-10-20 15:08:48,606 INFO [org.ovirt.engine.core.bll.UpdateVmPoolWithVmsCommand] (ajp-/127.0.0.1:8702-3) [7e5f56a9] Running command: UpdateVmPoolWithVmsCommand internal: false. Entities affected : ID: 8f1febe6-db76-4ff6-ab30-7f42cb8cf781 Type: VmPoolAction group EDIT_VM_POOL_CONFIGURATION with role type USER >2014-10-20 15:08:48,665 WARN [org.ovirt.engine.core.bll.AddVmAndAttachToPoolCommand] (ajp-/127.0.0.1:8702-3) [528941f0] CanDoAction of action AddVmAndAttachToPool failed. Reasons:VAR__ACTION__ADD,VAR__TYPE__VM,ACTION_TYPE_FAILED_ILLEGAL_OS_TYPE_IS_NOT_SUPPORTED_BY_ARCHITECTURE_TYPE >2014-10-20 15:08:48,680 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) [528941f0] Correlation ID: 7e5f56a9, Job ID: 7d0a26c4-20ca-461d-8468-a7ee18310cc5, Call Stack: null, Custom Event ID: -1, Message: Failed to update VM Pool ppc_pool(User: admin). >2014-10-20 15:08:48,692 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (ajp-/127.0.0.1:8702-3) Operation Failed: [Cannot edit VM-Pool. Selected operating system is not supported by the architecture.] >2014-10-20 15:08:48,712 INFO [org.ovirt.engine.core.bll.aaa.LogoutBySessionCommand] (ajp-/127.0.0.1:8702-3) [7f3571dc] Running command: LogoutBySessionCommand internal: false. >2014-10-20 15:08:48,717 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-3) [302f3c54] Running command: LogoutUserCommand internal: false. >2014-10-20 15:08:48,732 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) [302f3c54] Correlation ID: 302f3c54, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-10-20 15:09:07,449 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-8) [5c27910a] Setting new tasks map. The map contains now 3 tasks >2014-10-20 15:09:25,287 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-6) Running command: LoginUserCommand internal: false. >2014-10-20 15:09:25,290 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-10-20 15:09:37,450 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-23) Setting new tasks map. The map contains now 0 tasks >2014-10-20 15:09:37,450 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-23) Cleared all tasks of pool f8f5eaee-8fd0-4b45-87db-62d61b03a916. >2014-10-20 16:04:07,515 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-86) Deactivating non existing principal admin::internal >2014-10-20 16:04:08,446 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-90) Attempting to update VMs/Templates Ovf. >2014-10-20 16:04:09,100 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-90) [3ac2dccd] Lock Acquired to object EngineLock [exclusiveLocks= key: 1742b7af-bb81-4e6b-bf10-feab68a2f799 value: DISK >key: a1341057-9cf8-4750-a734-656d05a5d2c5 value: STORAGE >key: dfba228f-46f3-4758-a4f0-05857683a088 value: DISK >, sharedLocks= key: f8f5eaee-8fd0-4b45-87db-62d61b03a916 value: OVF_UPDATE >] >2014-10-20 16:04:09,104 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-90) [3ac2dccd] Running command: ProcessOvfUpdateForStorageDomainCommand internal: true. Entities affected : ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: Storage >2014-10-20 16:04:09,181 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-90) [3ac2dccd] START, SetVolumeDescriptionVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = dfba228f-46f3-4758-a4f0-05857683a088, imageId = 26155adf-9478-4927-8c49-8e12088c8648), log id: 615377ed >2014-10-20 16:04:09,182 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-90) [3ac2dccd] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-10-20 16:04:09,182 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-90) [3ac2dccd] -- setVolumeDescription parameters: > spUUID=f8f5eaee-8fd0-4b45-87db-62d61b03a916 > sdUUID=a1341057-9cf8-4750-a734-656d05a5d2c5 > imageGroupGUID=dfba228f-46f3-4758-a4f0-05857683a088 > volUUID=26155adf-9478-4927-8c49-8e12088c8648 > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"a1341057-9cf8-4750-a734-656d05a5d2c5"}],"Last Updated":"Sun Oct 19 13:59:30 IDT 2014"} > >2014-10-20 16:04:09,268 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-90) [3ac2dccd] FINISH, SetVolumeDescriptionVDSCommand, log id: 615377ed >2014-10-20 16:04:09,329 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-90) [6b08bb60] Running command: UploadStreamCommand internal: true. Entities affected : ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: Storage >2014-10-20 16:04:09,338 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-90) [6b08bb60] START, UploadStreamVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6), log id: 57412b8e >2014-10-20 16:04:09,338 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-90) [6b08bb60] -- executeVdsBrokerCommand: >2014-10-20 16:04:09,339 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-90) [6b08bb60] -- parameters: > spUUID=f8f5eaee-8fd0-4b45-87db-62d61b03a916 > sdUUID=a1341057-9cf8-4750-a734-656d05a5d2c5 > imageGUID=dfba228f-46f3-4758-a4f0-05857683a088 > volUUID=26155adf-9478-4927-8c49-8e12088c8648 > size=81,920 > >2014-10-20 16:04:11,369 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-90) [6b08bb60] FINISH, UploadStreamVDSCommand, log id: 57412b8e >2014-10-20 16:04:11,372 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-90) [6b08bb60] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 97cce701-2b52-4e28-9fc4-faba9b7842a8 >2014-10-20 16:04:11,372 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-90) [6b08bb60] CommandMultiAsyncTasks::AttachTask: Attaching task 35a84532-5568-4f08-9b5c-857b6d7383e3 to command 97cce701-2b52-4e28-9fc4-faba9b7842a8. >2014-10-20 16:04:11,382 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-90) [6b08bb60] Adding task 35a84532-5568-4f08-9b5c-857b6d7383e3 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-10-20 16:04:11,387 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-90) [6b08bb60] Lock freed to object EngineLock [exclusiveLocks= key: 1742b7af-bb81-4e6b-bf10-feab68a2f799 value: DISK >key: a1341057-9cf8-4750-a734-656d05a5d2c5 value: STORAGE >key: dfba228f-46f3-4758-a4f0-05857683a088 value: DISK >, sharedLocks= key: f8f5eaee-8fd0-4b45-87db-62d61b03a916 value: OVF_UPDATE >] >2014-10-20 16:04:11,393 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-90) [6b08bb60] START, SetVolumeDescriptionVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = dfba228f-46f3-4758-a4f0-05857683a088, imageId = 26155adf-9478-4927-8c49-8e12088c8648), log id: 4d84c844 >2014-10-20 16:04:11,394 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-90) [6b08bb60] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-10-20 16:04:11,394 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-90) [6b08bb60] -- setVolumeDescription parameters: > spUUID=f8f5eaee-8fd0-4b45-87db-62d61b03a916 > sdUUID=a1341057-9cf8-4750-a734-656d05a5d2c5 > imageGroupGUID=dfba228f-46f3-4758-a4f0-05857683a088 > volUUID=26155adf-9478-4927-8c49-8e12088c8648 > description={"Updated":true,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"a1341057-9cf8-4750-a734-656d05a5d2c5"}],"Last Updated":"Mon Oct 20 16:04:09 IDT 2014","Size":81920} > >2014-10-20 16:04:11,510 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-90) [6b08bb60] FINISH, SetVolumeDescriptionVDSCommand, log id: 4d84c844 >2014-10-20 16:04:11,517 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-90) [6b08bb60] START, SetVolumeDescriptionVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = 1742b7af-bb81-4e6b-bf10-feab68a2f799, imageId = 5b2b66c8-425d-4c49-9b3c-62f7586aac1b), log id: 14cddb08 >2014-10-20 16:04:11,525 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-90) [6b08bb60] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-10-20 16:04:11,525 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-90) [6b08bb60] -- setVolumeDescription parameters: > spUUID=f8f5eaee-8fd0-4b45-87db-62d61b03a916 > sdUUID=a1341057-9cf8-4750-a734-656d05a5d2c5 > imageGroupGUID=1742b7af-bb81-4e6b-bf10-feab68a2f799 > volUUID=5b2b66c8-425d-4c49-9b3c-62f7586aac1b > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"a1341057-9cf8-4750-a734-656d05a5d2c5"}],"Last Updated":"Sun Oct 19 13:59:30 IDT 2014"} > >2014-10-20 16:04:11,738 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-90) [6b08bb60] FINISH, SetVolumeDescriptionVDSCommand, log id: 14cddb08 >2014-10-20 16:04:11,797 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-90) [3551cc46] Running command: UploadStreamCommand internal: true. Entities affected : ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: Storage >2014-10-20 16:04:11,799 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-90) [3551cc46] START, UploadStreamVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6), log id: cbbe5e0 >2014-10-20 16:04:11,799 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-90) [3551cc46] -- executeVdsBrokerCommand: >2014-10-20 16:04:11,799 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-90) [3551cc46] -- parameters: > spUUID=f8f5eaee-8fd0-4b45-87db-62d61b03a916 > sdUUID=a1341057-9cf8-4750-a734-656d05a5d2c5 > imageGUID=1742b7af-bb81-4e6b-bf10-feab68a2f799 > volUUID=5b2b66c8-425d-4c49-9b3c-62f7586aac1b > size=81,920 > >2014-10-20 16:04:13,489 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-90) [3551cc46] FINISH, UploadStreamVDSCommand, log id: cbbe5e0 >2014-10-20 16:04:13,492 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-90) [3551cc46] CommandMultiAsyncTasks::AttachTask: Attaching task 36a0d5b9-dd2c-42d6-bb25-8aa23c7ee744 to command 97cce701-2b52-4e28-9fc4-faba9b7842a8. >2014-10-20 16:04:13,503 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-90) [3551cc46] Adding task 36a0d5b9-dd2c-42d6-bb25-8aa23c7ee744 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-10-20 16:04:13,508 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-90) [3551cc46] Trying to release a shared lock for key: f8f5eaee-8fd0-4b45-87db-62d61b03a916OVF_UPDATE , but lock does not exist >2014-10-20 16:04:13,509 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-90) [3551cc46] Trying to release exclusive lock which does not exist, lock key: 1742b7af-bb81-4e6b-bf10-feab68a2f799DISK >2014-10-20 16:04:13,509 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-90) [3551cc46] Trying to release exclusive lock which does not exist, lock key: a1341057-9cf8-4750-a734-656d05a5d2c5STORAGE >2014-10-20 16:04:13,509 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-90) [3551cc46] Trying to release exclusive lock which does not exist, lock key: dfba228f-46f3-4758-a4f0-05857683a088DISK >2014-10-20 16:04:13,509 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-90) [3551cc46] Lock freed to object EngineLock [exclusiveLocks= key: 1742b7af-bb81-4e6b-bf10-feab68a2f799 value: DISK >key: a1341057-9cf8-4750-a734-656d05a5d2c5 value: STORAGE >key: dfba228f-46f3-4758-a4f0-05857683a088 value: DISK >, sharedLocks= key: f8f5eaee-8fd0-4b45-87db-62d61b03a916 value: OVF_UPDATE >] >2014-10-20 16:04:13,510 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-90) [3551cc46] START, SetVolumeDescriptionVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = 1742b7af-bb81-4e6b-bf10-feab68a2f799, imageId = 5b2b66c8-425d-4c49-9b3c-62f7586aac1b), log id: 5e68eaae >2014-10-20 16:04:13,511 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-90) [3551cc46] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-10-20 16:04:13,511 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-90) [3551cc46] -- setVolumeDescription parameters: > spUUID=f8f5eaee-8fd0-4b45-87db-62d61b03a916 > sdUUID=a1341057-9cf8-4750-a734-656d05a5d2c5 > imageGroupGUID=1742b7af-bb81-4e6b-bf10-feab68a2f799 > volUUID=5b2b66c8-425d-4c49-9b3c-62f7586aac1b > description={"Updated":true,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"a1341057-9cf8-4750-a734-656d05a5d2c5"}],"Last Updated":"Mon Oct 20 16:04:09 IDT 2014","Size":81920} > >2014-10-20 16:04:13,624 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-90) [3551cc46] FINISH, SetVolumeDescriptionVDSCommand, log id: 5e68eaae >2014-10-20 16:04:13,631 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-90) [3551cc46] BaseAsyncTask::startPollingTask: Starting to poll task 35a84532-5568-4f08-9b5c-857b6d7383e3. >2014-10-20 16:04:13,631 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-90) [3551cc46] BaseAsyncTask::startPollingTask: Starting to poll task 36a0d5b9-dd2c-42d6-bb25-8aa23c7ee744. >2014-10-20 16:04:18,138 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-29) Polling and updating Async Tasks: 2 tasks, 2 tasks to poll now >2014-10-20 16:04:18,158 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-29) SPMAsyncTask::PollTask: Polling task 36a0d5b9-dd2c-42d6-bb25-8aa23c7ee744 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-10-20 16:04:18,158 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-29) BaseAsyncTask::onTaskEndSuccess: Task 36a0d5b9-dd2c-42d6-bb25-8aa23c7ee744 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-10-20 16:04:18,159 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-29) Task with DB Task ID 93ab2656-2d79-4001-b859-190a0fcb4793 and VDSM Task ID 35a84532-5568-4f08-9b5c-857b6d7383e3 is in state Polling. End action for command 97cce701-2b52-4e28-9fc4-faba9b7842a8 will proceed when all the entitys tasks are completed. >2014-10-20 16:04:18,159 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-29) SPMAsyncTask::PollTask: Polling task 35a84532-5568-4f08-9b5c-857b6d7383e3 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-10-20 16:04:18,159 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-29) BaseAsyncTask::onTaskEndSuccess: Task 35a84532-5568-4f08-9b5c-857b6d7383e3 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-10-20 16:04:18,159 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-29) CommandAsyncTask::endActionIfNecessary: All tasks of command 97cce701-2b52-4e28-9fc4-faba9b7842a8 has ended -> executing endAction >2014-10-20 16:04:18,159 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-29) CommandAsyncTask::endAction: Ending action for 2 tasks (command ID: 97cce701-2b52-4e28-9fc4-faba9b7842a8): calling endAction . >2014-10-20 16:04:18,160 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-19) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction ProcessOvfUpdateForStorageDomain, executionIndex: 0 >2014-10-20 16:04:18,221 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (org.ovirt.thread.pool-7-thread-19) [3ac2dccd] Ending command successfully: org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand >2014-10-20 16:04:18,232 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-19) [3ac2dccd] Trying to release a shared lock for key: f8f5eaee-8fd0-4b45-87db-62d61b03a916OVF_UPDATE , but lock does not exist >2014-10-20 16:04:18,244 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-19) [3ac2dccd] Trying to release exclusive lock which does not exist, lock key: 1742b7af-bb81-4e6b-bf10-feab68a2f799DISK >2014-10-20 16:04:18,244 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-19) [3ac2dccd] Trying to release exclusive lock which does not exist, lock key: a1341057-9cf8-4750-a734-656d05a5d2c5STORAGE >2014-10-20 16:04:18,244 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-19) [3ac2dccd] Trying to release exclusive lock which does not exist, lock key: dfba228f-46f3-4758-a4f0-05857683a088DISK >2014-10-20 16:04:18,244 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (org.ovirt.thread.pool-7-thread-19) [3ac2dccd] Lock freed to object EngineLock [exclusiveLocks= key: 1742b7af-bb81-4e6b-bf10-feab68a2f799 value: DISK >key: a1341057-9cf8-4750-a734-656d05a5d2c5 value: STORAGE >key: dfba228f-46f3-4758-a4f0-05857683a088 value: DISK >, sharedLocks= key: f8f5eaee-8fd0-4b45-87db-62d61b03a916 value: OVF_UPDATE >] >2014-10-20 16:04:18,244 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-19) [3ac2dccd] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type ProcessOvfUpdateForStorageDomain completed, handling the result. >2014-10-20 16:04:18,244 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-19) [3ac2dccd] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type ProcessOvfUpdateForStorageDomain succeeded, clearing tasks. >2014-10-20 16:04:18,244 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-19) [3ac2dccd] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 35a84532-5568-4f08-9b5c-857b6d7383e3 >2014-10-20 16:04:18,245 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-19) [3ac2dccd] START, SPMClearTaskVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, taskId = 35a84532-5568-4f08-9b5c-857b6d7383e3), log id: 59177ddb >2014-10-20 16:04:18,257 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-19) [3ac2dccd] START, HSMClearTaskVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, taskId=35a84532-5568-4f08-9b5c-857b6d7383e3), log id: 1547e28f >2014-10-20 16:04:18,288 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-19) [3ac2dccd] FINISH, HSMClearTaskVDSCommand, log id: 1547e28f >2014-10-20 16:04:18,288 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-19) [3ac2dccd] FINISH, SPMClearTaskVDSCommand, log id: 59177ddb >2014-10-20 16:04:18,292 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-19) [3ac2dccd] BaseAsyncTask::removeTaskFromDB: Removed task 35a84532-5568-4f08-9b5c-857b6d7383e3 from DataBase >2014-10-20 16:04:18,292 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-19) [3ac2dccd] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 36a0d5b9-dd2c-42d6-bb25-8aa23c7ee744 >2014-10-20 16:04:18,292 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-19) [3ac2dccd] START, SPMClearTaskVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, taskId = 36a0d5b9-dd2c-42d6-bb25-8aa23c7ee744), log id: 1956af88 >2014-10-20 16:04:18,294 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-19) [3ac2dccd] START, HSMClearTaskVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, taskId=36a0d5b9-dd2c-42d6-bb25-8aa23c7ee744), log id: a1e16b7 >2014-10-20 16:04:18,321 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-19) [3ac2dccd] FINISH, HSMClearTaskVDSCommand, log id: a1e16b7 >2014-10-20 16:04:18,322 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-19) [3ac2dccd] FINISH, SPMClearTaskVDSCommand, log id: 1956af88 >2014-10-20 16:04:18,326 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-19) [3ac2dccd] BaseAsyncTask::removeTaskFromDB: Removed task 36a0d5b9-dd2c-42d6-bb25-8aa23c7ee744 from DataBase >2014-10-20 16:04:18,326 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-19) [3ac2dccd] CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 97cce701-2b52-4e28-9fc4-faba9b7842a8 >2014-10-20 16:04:21,382 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-42) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 16:04:24,722 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-51) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 16:05:37,513 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-40) Setting new tasks map. The map contains now 0 tasks >2014-10-20 16:05:37,513 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-40) Cleared all tasks of pool f8f5eaee-8fd0-4b45-87db-62d61b03a916. >2014-10-20 16:58:39,340 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-69) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 17:04:13,632 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-27) Attempting to update VMs/Templates Ovf. >2014-10-20 17:04:23,662 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-88) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 17:04:33,560 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-15) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 17:58:53,650 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-30) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 18:04:13,667 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-49) Attempting to update VMs/Templates Ovf. >2014-10-20 18:04:25,955 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-98) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 18:04:43,942 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-65) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 18:59:06,977 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-79) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 19:04:13,693 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-65) Attempting to update VMs/Templates Ovf. >2014-10-20 19:04:26,215 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-1) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 19:04:54,032 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-19) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 418 seconds while maximum configured value is 300 seconds. >2014-10-20 19:59:21,650 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-72) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 20:04:13,726 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-64) Attempting to update VMs/Templates Ovf. >2014-10-20 20:04:27,033 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-16) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 418 seconds while maximum configured value is 300 seconds. >2014-10-20 20:05:01,489 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) [1883f47f] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 20:59:22,441 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-68) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 21:04:13,761 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-59) Attempting to update VMs/Templates Ovf. >2014-10-20 21:04:41,817 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-84) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 21:05:09,074 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-84) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 21:59:36,018 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-26) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has time-drift of 418 seconds while maximum configured value is 300 seconds. >2014-10-20 22:04:13,780 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-96) Attempting to update VMs/Templates Ovf. >2014-10-20 22:04:56,300 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-45) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 22:05:17,804 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-28) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 22:59:49,889 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-82) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 23:04:13,798 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-82) Attempting to update VMs/Templates Ovf. >2014-10-20 23:05:11,256 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-20 23:05:24,395 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-34) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 00:00:03,427 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-76) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 00:04:13,832 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-32) [29188d83] Attempting to update VMs/Templates Ovf. >2014-10-21 00:05:25,985 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-84) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 00:05:31,044 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-14) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 418 seconds while maximum configured value is 300 seconds. >2014-10-21 01:00:16,829 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 01:04:13,865 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-40) Attempting to update VMs/Templates Ovf. >2014-10-21 01:05:26,225 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-42) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 01:05:39,284 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-66) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 02:00:30,956 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-70) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 02:04:13,889 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-33) [75e80df5] Attempting to update VMs/Templates Ovf. >2014-10-21 02:05:41,335 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-88) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 02:05:46,843 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-5) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 03:00:44,631 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-100) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 03:04:13,904 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-43) Attempting to update VMs/Templates Ovf. >2014-10-21 03:05:53,553 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 03:05:56,288 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-72) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 03:35:35,001 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-80) Start deleteAgedOutAuditLogs >2014-10-21 03:35:35,001 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-34) Start deleteAgedOutCommandEntities >2014-10-21 03:35:35,015 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-80) Finished deleteAgedOutAuditLogs >2014-10-21 03:35:35,016 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-34) Finished deleteAgedOutCommandEntities >2014-10-21 04:00:58,175 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-22) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 04:04:13,920 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-91) Attempting to update VMs/Templates Ovf. >2014-10-21 04:05:56,515 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-66) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 04:06:03,114 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-86) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 05:01:11,829 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-92) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 05:04:13,935 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-67) Attempting to update VMs/Templates Ovf. >2014-10-21 05:06:11,513 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-62) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 05:06:12,213 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-22) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 06:01:24,909 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-87) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 06:04:13,957 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-17) Attempting to update VMs/Templates Ovf. >2014-10-21 06:06:20,097 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-54) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 06:06:26,565 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-89) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 07:01:37,823 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-21) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 07:04:13,974 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-30) Attempting to update VMs/Templates Ovf. >2014-10-21 07:06:27,057 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-99) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 07:06:41,528 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-68) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 08:01:51,075 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-75) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 08:04:13,990 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-13) Attempting to update VMs/Templates Ovf. >2014-10-21 08:06:35,858 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-85) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 08:06:41,733 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-41) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 09:02:04,692 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-71) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 09:04:14,006 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-35) Attempting to update VMs/Templates Ovf. >2014-10-21 09:06:43,850 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-73) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 09:06:56,587 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-78) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 09:19:53,210 INFO [org.ovirt.engine.ui.frontend.server.gwt.plugin.PluginDataManager] (ajp-/127.0.0.1:8702-7) Reading UI plugin descriptor [/usr/share/ovirt-engine/ui-plugins/redhat_support_plugin_rhev.json] >2014-10-21 09:19:53,211 INFO [org.ovirt.engine.ui.frontend.server.gwt.plugin.PluginDataManager] (ajp-/127.0.0.1:8702-7) Reading UI plugin configuration [/etc/ovirt-engine/ui-plugins/redhat_support_plugin_rhev-config.json] >2014-10-21 09:19:53,222 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-7) Invalid locale found in configuration: >2014-10-21 09:19:53,222 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-7) Invalid locale found in configuration: >2014-10-21 09:20:00,393 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (ajp-/127.0.0.1:8702-7) Running command: LoginAdminUserCommand internal: false. >2014-10-21 09:20:00,401 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-10-21 09:20:01,679 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-1) Running command: LoginUserCommand internal: false. >2014-10-21 10:02:17,924 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-14) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 10:04:07,569 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-40) Deactivating non existing principal admin::internal >2014-10-21 10:04:14,022 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-83) Attempting to update VMs/Templates Ovf. >2014-10-21 10:06:52,144 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-38) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 10:06:57,596 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-41) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 11:02:31,973 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-54) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 11:04:14,039 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-85) Attempting to update VMs/Templates Ovf. >2014-10-21 11:06:57,722 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-18) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 11:07:01,215 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-92) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 12:02:46,295 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-37) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 12:04:14,055 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-72) Attempting to update VMs/Templates Ovf. >2014-10-21 12:06:58,633 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-89) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 12:07:08,610 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-51) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 13:03:00,623 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-25) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 13:04:14,072 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-5) Attempting to update VMs/Templates Ovf. >2014-10-21 13:07:02,156 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-46) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 13:07:18,677 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-70) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 14:03:14,936 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-80) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 14:04:14,089 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-82) Attempting to update VMs/Templates Ovf. >2014-10-21 14:07:02,512 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-35) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 14:07:25,930 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-47) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 15:03:29,681 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 15:04:14,102 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-42) Attempting to update VMs/Templates Ovf. >2014-10-21 15:07:03,045 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-100) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 418 seconds while maximum configured value is 300 seconds. >2014-10-21 15:07:35,536 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 15:11:16,372 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-7) Invalid locale found in configuration: >2014-10-21 15:11:16,372 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-7) Invalid locale found in configuration: >2014-10-21 15:11:19,997 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-10) Invalid locale found in configuration: >2014-10-21 15:11:19,998 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-10) Invalid locale found in configuration: >2014-10-21 15:11:30,237 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (ajp-/127.0.0.1:8702-1) Running command: LoginAdminUserCommand internal: false. >2014-10-21 15:11:30,242 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-10-21 15:11:30,948 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-4) Running command: LoginUserCommand internal: false. >2014-10-21 15:12:39,803 INFO [org.ovirt.engine.core.bll.storage.AddStorageServerConnectionCommand] (ajp-/127.0.0.1:8702-3) [ea12841] Lock Acquired to object EngineLock [exclusiveLocks= key: 10.35.160.108:/RHEV/mom_export value: STORAGE_CONNECTION >, sharedLocks= ] >2014-10-21 15:12:39,828 INFO [org.ovirt.engine.core.bll.storage.AddStorageServerConnectionCommand] (ajp-/127.0.0.1:8702-3) [ea12841] Running command: AddStorageServerConnectionCommand internal: false. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-10-21 15:12:39,830 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (ajp-/127.0.0.1:8702-3) [ea12841] START, ConnectStorageServerVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: null, connection: 10.35.160.108:/RHEV/mom_export, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 1a23d2d8 >2014-10-21 15:12:40,026 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (ajp-/127.0.0.1:8702-3) [ea12841] FINISH, ConnectStorageServerVDSCommand, return: {00000000-0000-0000-0000-000000000000=0}, log id: 1a23d2d8 >2014-10-21 15:12:40,044 INFO [org.ovirt.engine.core.bll.storage.AddStorageServerConnectionCommand] (ajp-/127.0.0.1:8702-3) [ea12841] Lock freed to object EngineLock [exclusiveLocks= key: 10.35.160.108:/RHEV/mom_export value: STORAGE_CONNECTION >, sharedLocks= ] >2014-10-21 15:12:40,402 INFO [org.ovirt.engine.core.bll.storage.GetExistingStorageDomainListQuery] (ajp-/127.0.0.1:8702-11) START, GetExistingStorageDomainListQuery(refresh: true, filtered: false), log id: 6ab8db15 >2014-10-21 15:12:40,406 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetStorageDomainsListVDSCommand] (ajp-/127.0.0.1:8702-11) START, HSMGetStorageDomainsListVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, storagePoolId=00000000-0000-0000-0000-000000000000, storageType=NFS, storageDomainType=ImportExport, path=10.35.160.108:/RHEV/mom_export), log id: 2ba794cc >2014-10-21 15:12:41,162 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetStorageDomainsListVDSCommand] (ajp-/127.0.0.1:8702-11) FINISH, HSMGetStorageDomainsListVDSCommand, return: [76b88f40-ff55-49c3-921c-64e0a4c14418], log id: 2ba794cc >2014-10-21 15:12:41,256 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetStorageDomainInfoVDSCommand] (ajp-/127.0.0.1:8702-11) START, HSMGetStorageDomainInfoVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, storageDomainId=76b88f40-ff55-49c3-921c-64e0a4c14418), log id: 3c157184 >2014-10-21 15:12:41,301 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetStorageDomainInfoVDSCommand] (ajp-/127.0.0.1:8702-11) FINISH, HSMGetStorageDomainInfoVDSCommand, return: <StorageDomainStatic[mom_export, 76b88f40-ff55-49c3-921c-64e0a4c14418], fef4af93-a222-4206-b33c-3c4112047019>, log id: 3c157184 >2014-10-21 15:12:41,302 INFO [org.ovirt.engine.core.bll.storage.GetExistingStorageDomainListQuery] (ajp-/127.0.0.1:8702-11) FINISH, GetExistingStorageDomainListQuery, log id: 6ab8db15 >2014-10-21 15:12:41,966 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetStorageDomainsListVDSCommand] (ajp-/127.0.0.1:8702-1) [31fc4dc7] START, HSMGetStorageDomainsListVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, storagePoolId=00000000-0000-0000-0000-000000000000, storageType=NFS, storageDomainType=ImportExport, path=), log id: 7fc87d3f >2014-10-21 15:12:42,554 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetStorageDomainsListVDSCommand] (ajp-/127.0.0.1:8702-1) [31fc4dc7] FINISH, HSMGetStorageDomainsListVDSCommand, return: [76b88f40-ff55-49c3-921c-64e0a4c14418], log id: 7fc87d3f >2014-10-21 15:12:42,556 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetStorageDomainInfoVDSCommand] (ajp-/127.0.0.1:8702-1) [31fc4dc7] START, HSMGetStorageDomainInfoVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, storageDomainId=76b88f40-ff55-49c3-921c-64e0a4c14418), log id: 3d4e9702 >2014-10-21 15:12:42,577 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetStorageDomainInfoVDSCommand] (ajp-/127.0.0.1:8702-1) [31fc4dc7] FINISH, HSMGetStorageDomainInfoVDSCommand, return: <StorageDomainStatic[mom_export, 76b88f40-ff55-49c3-921c-64e0a4c14418], fef4af93-a222-4206-b33c-3c4112047019>, log id: 3d4e9702 >2014-10-21 15:12:42,584 INFO [org.ovirt.engine.core.bll.storage.AddExistingFileStorageDomainCommand] (ajp-/127.0.0.1:8702-1) [31fc4dc7] Running command: AddExistingFileStorageDomainCommand internal: false. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-10-21 15:12:42,598 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStorageDomainStatsVDSCommand] (ajp-/127.0.0.1:8702-1) [31fc4dc7] START, GetStorageDomainStatsVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, storageDomainId=76b88f40-ff55-49c3-921c-64e0a4c14418), log id: 2d66ffe8 >2014-10-21 15:12:42,612 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStorageDomainStatsVDSCommand] (ajp-/127.0.0.1:8702-1) [31fc4dc7] FINISH, GetStorageDomainStatsVDSCommand, return: StorageDomain[, 76b88f40-ff55-49c3-921c-64e0a4c14418], log id: 2d66ffe8 >2014-10-21 15:12:42,643 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [31fc4dc7] Correlation ID: 31fc4dc7, Job ID: 72050540-c7df-4905-93d7-e62e1b3cda98, Call Stack: null, Custom Event ID: -1, Message: Storage Domain mom_export was added by admin >2014-10-21 15:12:43,131 INFO [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] (ajp-/127.0.0.1:8702-10) [63d238c6] Lock Acquired to object EngineLock [exclusiveLocks= key: 76b88f40-ff55-49c3-921c-64e0a4c14418 value: STORAGE >, sharedLocks= ] >2014-10-21 15:12:43,168 INFO [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] (ajp-/127.0.0.1:8702-10) [63d238c6] Running command: AttachStorageDomainToPoolCommand internal: false. Entities affected : ID: 76b88f40-ff55-49c3-921c-64e0a4c14418 Type: StorageAction group MANIPULATE_STORAGE_DOMAIN with role type ADMIN >2014-10-21 15:12:43,199 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (org.ovirt.thread.pool-7-thread-1) [de3e6ff] Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-10-21 15:12:43,200 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (org.ovirt.thread.pool-7-thread-15) [200ea263] Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-10-21 15:12:43,203 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-1) [de3e6ff] START, ConnectStorageServerVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: a4d5f882-3b1b-43ad-9ccb-cca623ff7d1b, connection: 10.35.160.108:/RHEV/mom_export, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 6c8ccb03 >2014-10-21 15:12:43,205 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-15) [200ea263] START, ConnectStorageServerVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: a4d5f882-3b1b-43ad-9ccb-cca623ff7d1b, connection: 10.35.160.108:/RHEV/mom_export, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 260631e5 >2014-10-21 15:12:43,206 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (org.ovirt.thread.pool-7-thread-22) [6d527678] Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-10-21 15:12:43,255 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-22) [6d527678] START, ConnectStorageServerVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: a4d5f882-3b1b-43ad-9ccb-cca623ff7d1b, connection: 10.35.160.108:/RHEV/mom_export, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 36cb64bb >2014-10-21 15:12:43,255 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-1) [de3e6ff] FINISH, ConnectStorageServerVDSCommand, return: {a4d5f882-3b1b-43ad-9ccb-cca623ff7d1b=0}, log id: 6c8ccb03 >2014-10-21 15:12:43,288 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-15) [200ea263] FINISH, ConnectStorageServerVDSCommand, return: {a4d5f882-3b1b-43ad-9ccb-cca623ff7d1b=0}, log id: 260631e5 >2014-10-21 15:12:43,373 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-22) [6d527678] FINISH, ConnectStorageServerVDSCommand, return: {a4d5f882-3b1b-43ad-9ccb-cca623ff7d1b=0}, log id: 36cb64bb >2014-10-21 15:12:43,377 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand] (ajp-/127.0.0.1:8702-10) [63d238c6] START, AttachStorageDomainVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = 76b88f40-ff55-49c3-921c-64e0a4c14418), log id: 63ef58e9 >2014-10-21 15:13:04,869 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand] (ajp-/127.0.0.1:8702-10) [63d238c6] Command AttachStorageDomainVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = 76b88f40-ff55-49c3-921c-64e0a4c14418) execution failed. Exception: IrsOperationFailedNoFailoverException: IRSGenericException: IRSErrorException: Storage domain already attached to pool: u'domain=76b88f40-ff55-49c3-921c-64e0a4c14418, pool=fef4af93-a222-4206-b33c-3c4112047019' >2014-10-21 15:13:04,869 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand] (ajp-/127.0.0.1:8702-10) [63d238c6] FINISH, AttachStorageDomainVDSCommand, log id: 63ef58e9 >2014-10-21 15:13:04,869 ERROR [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] (ajp-/127.0.0.1:8702-10) [63d238c6] Command org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand throw Vdc Bll exception. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.irsbroker.IrsOperationFailedNoFailoverException: IRSGenericException: IRSErrorException: Storage domain already attached to pool: u'domain=76b88f40-ff55-49c3-921c-64e0a4c14418, pool=fef4af93-a222-4206-b33c-3c4112047019' (Failed with error StorageDomainAlreadyAttached and code 380) >2014-10-21 15:13:04,917 INFO [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] (ajp-/127.0.0.1:8702-10) [63d238c6] Command [id=e3fa6328-096a-4a65-b7cc-a466e7773018]: Compensating NEW_ENTITY_ID of org.ovirt.engine.core.common.businessentities.StoragePoolIsoMap; snapshot: storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, storageId = 76b88f40-ff55-49c3-921c-64e0a4c14418. >2014-10-21 15:13:04,982 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) [63d238c6] Correlation ID: 63d238c6, Job ID: 653947c0-e85a-4926-86b0-4974f1a69a3e, Call Stack: null, Custom Event ID: -1, Message: Failed to attach Storage Domain mom_export to Data Center dc_35. (User: admin) >2014-10-21 15:13:04,991 INFO [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] (ajp-/127.0.0.1:8702-10) [63d238c6] Lock freed to object EngineLock [exclusiveLocks= key: 76b88f40-ff55-49c3-921c-64e0a4c14418 value: STORAGE >, sharedLocks= ] >2014-10-21 15:17:35,426 INFO [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] (ajp-/127.0.0.1:8702-7) [57f14db2] Lock Acquired to object EngineLock [exclusiveLocks= key: 76b88f40-ff55-49c3-921c-64e0a4c14418 value: STORAGE >, sharedLocks= ] >2014-10-21 15:17:35,477 INFO [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] (org.ovirt.thread.pool-7-thread-13) [57f14db2] Running command: AttachStorageDomainToPoolCommand internal: false. Entities affected : ID: 76b88f40-ff55-49c3-921c-64e0a4c14418 Type: StorageAction group MANIPULATE_STORAGE_DOMAIN with role type ADMIN >2014-10-21 15:17:35,488 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (org.ovirt.thread.pool-7-thread-48) [33638992] Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-10-21 15:17:35,488 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (org.ovirt.thread.pool-7-thread-21) [4bdcea99] Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-10-21 15:17:35,491 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-21) [4bdcea99] START, ConnectStorageServerVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: a4d5f882-3b1b-43ad-9ccb-cca623ff7d1b, connection: 10.35.160.108:/RHEV/mom_export, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 75098bb8 >2014-10-21 15:17:35,491 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-48) [33638992] START, ConnectStorageServerVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: a4d5f882-3b1b-43ad-9ccb-cca623ff7d1b, connection: 10.35.160.108:/RHEV/mom_export, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 32a70c12 >2014-10-21 15:17:35,495 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (org.ovirt.thread.pool-7-thread-29) [4b00eaff] Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-10-21 15:17:35,503 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-29) [4b00eaff] START, ConnectStorageServerVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: a4d5f882-3b1b-43ad-9ccb-cca623ff7d1b, connection: 10.35.160.108:/RHEV/mom_export, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 1d383d37 >2014-10-21 15:17:35,505 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-21) [4bdcea99] FINISH, ConnectStorageServerVDSCommand, return: {a4d5f882-3b1b-43ad-9ccb-cca623ff7d1b=0}, log id: 75098bb8 >2014-10-21 15:17:35,507 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-48) [33638992] FINISH, ConnectStorageServerVDSCommand, return: {a4d5f882-3b1b-43ad-9ccb-cca623ff7d1b=0}, log id: 32a70c12 >2014-10-21 15:17:35,513 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-29) [4b00eaff] FINISH, ConnectStorageServerVDSCommand, return: {a4d5f882-3b1b-43ad-9ccb-cca623ff7d1b=0}, log id: 1d383d37 >2014-10-21 15:17:35,515 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand] (org.ovirt.thread.pool-7-thread-13) [57f14db2] START, AttachStorageDomainVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = 76b88f40-ff55-49c3-921c-64e0a4c14418), log id: 294513fc >2014-10-21 15:17:57,013 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand] (org.ovirt.thread.pool-7-thread-13) [57f14db2] FINISH, AttachStorageDomainVDSCommand, log id: 294513fc >2014-10-21 15:17:57,059 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImagesListVDSCommand] (org.ovirt.thread.pool-7-thread-13) [57f14db2] START, GetImagesListVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, sdUUID = 76b88f40-ff55-49c3-921c-64e0a4c14418), log id: 7a74d017 >2014-10-21 15:17:57,110 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImagesListVDSCommand] (org.ovirt.thread.pool-7-thread-13) [57f14db2] FINISH, GetImagesListVDSCommand, return: [0d3417e9-f9c2-46c7-a752-92a697c7e48d, cae73011-e89d-434b-91b6-c79e60e3df42, c82a4d1a-1a37-4a07-b6dd-493a2418b911], log id: 7a74d017 >2014-10-21 15:17:57,130 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVolumesListVDSCommand] (org.ovirt.thread.pool-7-thread-13) [57f14db2] START, GetVolumesListVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = 76b88f40-ff55-49c3-921c-64e0a4c14418, imageGroupId = 0d3417e9-f9c2-46c7-a752-92a697c7e48d), log id: 1d5775ea >2014-10-21 15:17:57,147 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVolumesListVDSCommand] (org.ovirt.thread.pool-7-thread-13) [57f14db2] FINISH, GetVolumesListVDSCommand, return: [8d249ece-5b0b-4fae-ba9d-58e0188111bc], log id: 1d5775ea >2014-10-21 15:17:57,148 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-13) [57f14db2] START, GetImageInfoVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = 76b88f40-ff55-49c3-921c-64e0a4c14418, imageGroupId = 0d3417e9-f9c2-46c7-a752-92a697c7e48d, imageId = 8d249ece-5b0b-4fae-ba9d-58e0188111bc), log id: 5a297c28 >2014-10-21 15:17:57,192 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-13) [57f14db2] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@bd5b5e5d, log id: 5a297c28 >2014-10-21 15:17:57,224 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVolumesListVDSCommand] (org.ovirt.thread.pool-7-thread-13) [57f14db2] START, GetVolumesListVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = 76b88f40-ff55-49c3-921c-64e0a4c14418, imageGroupId = cae73011-e89d-434b-91b6-c79e60e3df42), log id: 20d0f676 >2014-10-21 15:17:57,240 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVolumesListVDSCommand] (org.ovirt.thread.pool-7-thread-13) [57f14db2] FINISH, GetVolumesListVDSCommand, return: [51aa1ba0-792e-4a6e-8ec5-29e6f1d68d3a], log id: 20d0f676 >2014-10-21 15:17:57,240 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-13) [57f14db2] START, GetImageInfoVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = 76b88f40-ff55-49c3-921c-64e0a4c14418, imageGroupId = cae73011-e89d-434b-91b6-c79e60e3df42, imageId = 51aa1ba0-792e-4a6e-8ec5-29e6f1d68d3a), log id: 76e4cdce >2014-10-21 15:17:57,403 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-13) [57f14db2] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@e9dc3e47, log id: 76e4cdce >2014-10-21 15:17:57,413 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVolumesListVDSCommand] (org.ovirt.thread.pool-7-thread-13) [57f14db2] START, GetVolumesListVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = 76b88f40-ff55-49c3-921c-64e0a4c14418, imageGroupId = c82a4d1a-1a37-4a07-b6dd-493a2418b911), log id: 58dcffe9 >2014-10-21 15:17:57,473 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVolumesListVDSCommand] (org.ovirt.thread.pool-7-thread-13) [57f14db2] FINISH, GetVolumesListVDSCommand, return: [2a16b9e0-a597-4e1c-848c-a536c21278ac], log id: 58dcffe9 >2014-10-21 15:17:57,473 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-13) [57f14db2] START, GetImageInfoVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = 76b88f40-ff55-49c3-921c-64e0a4c14418, imageGroupId = c82a4d1a-1a37-4a07-b6dd-493a2418b911, imageId = 2a16b9e0-a597-4e1c-848c-a536c21278ac), log id: 15254793 >2014-10-21 15:17:57,556 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-13) [57f14db2] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@65162ae4, log id: 15254793 >2014-10-21 15:17:57,557 WARN [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] (org.ovirt.thread.pool-7-thread-13) There are no OVF_STORE disks on storage domain id 76b88f40-ff55-49c3-921c-64e0a4c14418 >2014-10-21 15:17:57,564 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-7-thread-13) [49eeb01] Failed to Acquire Lock to object EngineLock [exclusiveLocks= key: 76b88f40-ff55-49c3-921c-64e0a4c14418 value: STORAGE >, sharedLocks= ] >2014-10-21 15:17:57,564 WARN [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-7-thread-13) [49eeb01] CanDoAction of action ActivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__ACTIVATE,ACTION_TYPE_FAILED_OBJECT_LOCKED >2014-10-21 15:17:57,578 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-13) [49eeb01] Correlation ID: 57f14db2, Job ID: c64fd2c2-443c-4c91-a71c-f2ace7b03656, Call Stack: null, Custom Event ID: -1, Message: Storage Domain mom_export was attached to Data Center dc_35 by admin >2014-10-21 15:17:57,583 INFO [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] (org.ovirt.thread.pool-7-thread-13) [49eeb01] Lock freed to object EngineLock [exclusiveLocks= key: 76b88f40-ff55-49c3-921c-64e0a4c14418 value: STORAGE >, sharedLocks= ] >2014-10-21 15:17:57,595 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-7-thread-34) Adding domain 76b88f40-ff55-49c3-921c-64e0a4c14418 to the domains in maintenance cache >2014-10-21 15:18:30,277 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (ajp-/127.0.0.1:8702-2) [4f65715b] Lock Acquired to object EngineLock [exclusiveLocks= key: 76b88f40-ff55-49c3-921c-64e0a4c14418 value: STORAGE >, sharedLocks= ] >2014-10-21 15:18:30,308 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-7-thread-13) [4f65715b] Running command: ActivateStorageDomainCommand internal: false. Entities affected : ID: 76b88f40-ff55-49c3-921c-64e0a4c14418 Type: StorageAction group MANIPULATE_STORAGE_DOMAIN with role type ADMIN >2014-10-21 15:18:30,314 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-7-thread-13) [4f65715b] Lock freed to object EngineLock [exclusiveLocks= key: 76b88f40-ff55-49c3-921c-64e0a4c14418 value: STORAGE >, sharedLocks= ] >2014-10-21 15:18:30,315 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-7-thread-13) [4f65715b] ActivateStorage Domain. Before Connect all hosts to pool. Time:10/21/14 3:18 PM >2014-10-21 15:18:30,322 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (org.ovirt.thread.pool-7-thread-34) [6e006461] Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-10-21 15:18:30,322 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (org.ovirt.thread.pool-7-thread-35) [2caaa12c] Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-10-21 15:18:30,325 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (org.ovirt.thread.pool-7-thread-2) [675ac339] Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-10-21 15:18:30,326 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-34) [6e006461] START, ConnectStorageServerVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: a4d5f882-3b1b-43ad-9ccb-cca623ff7d1b, connection: 10.35.160.108:/RHEV/mom_export, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 63a506b2 >2014-10-21 15:18:30,329 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-2) [675ac339] START, ConnectStorageServerVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: a4d5f882-3b1b-43ad-9ccb-cca623ff7d1b, connection: 10.35.160.108:/RHEV/mom_export, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 42f415c3 >2014-10-21 15:18:30,330 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-35) [2caaa12c] START, ConnectStorageServerVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: a4d5f882-3b1b-43ad-9ccb-cca623ff7d1b, connection: 10.35.160.108:/RHEV/mom_export, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 29d43f02 >2014-10-21 15:18:30,339 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-35) [2caaa12c] FINISH, ConnectStorageServerVDSCommand, return: {a4d5f882-3b1b-43ad-9ccb-cca623ff7d1b=0}, log id: 29d43f02 >2014-10-21 15:18:30,340 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-2) [675ac339] FINISH, ConnectStorageServerVDSCommand, return: {a4d5f882-3b1b-43ad-9ccb-cca623ff7d1b=0}, log id: 42f415c3 >2014-10-21 15:18:30,344 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-34) [6e006461] FINISH, ConnectStorageServerVDSCommand, return: {a4d5f882-3b1b-43ad-9ccb-cca623ff7d1b=0}, log id: 63a506b2 >2014-10-21 15:18:30,346 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (org.ovirt.thread.pool-7-thread-13) [4f65715b] START, ActivateStorageDomainVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = 76b88f40-ff55-49c3-921c-64e0a4c14418), log id: 5d92034 >2014-10-21 15:18:30,645 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (org.ovirt.thread.pool-7-thread-13) [4f65715b] FINISH, ActivateStorageDomainVDSCommand, return: true, log id: 5d92034 >2014-10-21 15:18:30,646 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-7-thread-13) [4f65715b] ActivateStorage Domain. After Connect all hosts to pool. Time:10/21/14 3:18 PM >2014-10-21 15:18:30,662 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-7-thread-24) START, ConnectStoragePoolVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, vdsId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, masterVersion = 1), log id: 5108421a >2014-10-21 15:18:30,686 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-7-thread-3) START, ConnectStoragePoolVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vdsId = d5b2492d-8110-4614-9ce4-4a7a118c7942, storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, masterVersion = 1), log id: 679bc52f >2014-10-21 15:18:30,688 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-7-thread-26) START, ConnectStoragePoolVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vdsId = 1c39bd88-8672-408f-a593-60d6ef8f647c, storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, masterVersion = 1), log id: 4b993e9b >2014-10-21 15:18:31,018 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-7-thread-3) FINISH, ConnectStoragePoolVDSCommand, log id: 679bc52f >2014-10-21 15:18:31,018 INFO [org.ovirt.engine.core.bll.storage.RefreshPoolSingleAsyncOperation] (org.ovirt.thread.pool-7-thread-3) Refreshed vds alma07.qa.lab.tlv.redhat.com in pool dc_35 >2014-10-21 15:18:31,033 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-7-thread-26) FINISH, ConnectStoragePoolVDSCommand, log id: 4b993e9b >2014-10-21 15:18:31,033 INFO [org.ovirt.engine.core.bll.storage.RefreshPoolSingleAsyncOperation] (org.ovirt.thread.pool-7-thread-26) Refreshed vds alma06.qa.lab.tlv.redhat.com in pool dc_35 >2014-10-21 15:18:31,311 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-7-thread-24) FINISH, ConnectStoragePoolVDSCommand, log id: 5108421a >2014-10-21 15:18:31,311 INFO [org.ovirt.engine.core.bll.storage.RefreshPoolSingleAsyncOperation] (org.ovirt.thread.pool-7-thread-24) Refreshed vds cyan-vdsg.qa.lab.tlv.redhat.com in pool dc_35 >2014-10-21 15:18:31,312 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-7-thread-13) [4f65715b] ActivateStorage Domain. After change storage pool status in vds. Time:10/21/14 3:18 PM >2014-10-21 15:18:31,326 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-13) [4f65715b] Correlation ID: 4f65715b, Job ID: 3790212e-0290-480c-aadf-5d929b6399dd, Call Stack: null, Custom Event ID: -1, Message: Storage Domain mom_export (Data Center dc_35) was activated by admin >2014-10-21 15:18:44,420 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-7-thread-32) Removing domain 76b88f40-ff55-49c3-921c-64e0a4c14418 from the domains in maintenance cache >2014-10-21 15:19:33,150 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVmsInfoVDSCommand] (ajp-/127.0.0.1:8702-8) START, GetVmsInfoVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = 76b88f40-ff55-49c3-921c-64e0a4c14418, vmIdList = null), log id: 676006f7 >2014-10-21 15:19:33,218 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVmsInfoVDSCommand] (ajp-/127.0.0.1:8702-8) FINISH, GetVmsInfoVDSCommand, log id: 676006f7 >2014-10-21 15:20:08,426 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVmsInfoVDSCommand] (ajp-/127.0.0.1:8702-3) START, GetVmsInfoVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = 76b88f40-ff55-49c3-921c-64e0a4c14418, vmIdList = null), log id: 3390aee4 >2014-10-21 15:20:08,458 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVmsInfoVDSCommand] (ajp-/127.0.0.1:8702-3) FINISH, GetVmsInfoVDSCommand, log id: 3390aee4 >2014-10-21 15:20:20,431 INFO [org.ovirt.engine.core.bll.ImportVmCommand] (ajp-/127.0.0.1:8702-7) [32126abb] Lock Acquired to object EngineLock [exclusiveLocks= key: windows7 value: VM_NAME >, sharedLocks= key: ce03198a-a0c4-4745-b442-cf2c5eaeea45 value: REMOTE_VM >] >2014-10-21 15:20:20,443 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVmsInfoVDSCommand] (ajp-/127.0.0.1:8702-7) [32126abb] START, GetVmsInfoVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = 76b88f40-ff55-49c3-921c-64e0a4c14418, vmIdList = null), log id: 6f0ea005 >2014-10-21 15:20:20,470 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVmsInfoVDSCommand] (ajp-/127.0.0.1:8702-7) [32126abb] FINISH, GetVmsInfoVDSCommand, log id: 6f0ea005 >2014-10-21 15:20:21,304 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImagesListVDSCommand] (ajp-/127.0.0.1:8702-7) [32126abb] START, GetImagesListVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, sdUUID = a1341057-9cf8-4750-a734-656d05a5d2c5), log id: 2df4619e >2014-10-21 15:20:21,362 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImagesListVDSCommand] (ajp-/127.0.0.1:8702-7) [32126abb] FINISH, GetImagesListVDSCommand, return: [416239b4-ea7f-445b-a347-a4b44aa901b4, b12fe15d-81b9-4fa2-94c3-b7b2f90f1b4d, 2e7089ff-2fca-4243-b58b-dcef3678ca02, 9a68745c-3578-46d7-ad9c-d9e03c4639dd, 493ae1d9-5bb7-4948-a637-3e5bc861ac45, 65766f65-44ba-4e98-9e73-3d1fea2106e4, f988c27d-db38-4446-b082-2c4107280333, ff51fcdc-f313-49ef-a677-dfa8cbc00820, dfba228f-46f3-4758-a4f0-05857683a088, 3df49b50-c82d-42d0-b656-9877ca444e01, 3778d539-c7de-48fc-be30-9a02bf87665a, 2cc3f4d2-4b42-49b6-9260-5b604d5ffa81, 075b5199-62ff-44cd-8b8e-e1eccedfcb6c, 4323cfe2-a20f-48d0-8d3c-d02e52d43625, 40c0302a-f7f6-49a2-97a7-3aee6b07d502, 1742b7af-bb81-4e6b-bf10-feab68a2f799], log id: 2df4619e >2014-10-21 15:20:21,363 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DoesImageExistVDSCommand] (ajp-/127.0.0.1:8702-7) [32126abb] START, DoesImageExistVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = 76b88f40-ff55-49c3-921c-64e0a4c14418, imageGroupId = cae73011-e89d-434b-91b6-c79e60e3df42, imageId = 51aa1ba0-792e-4a6e-8ec5-29e6f1d68d3a), log id: 256c0e33 >2014-10-21 15:20:21,410 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DoesImageExistVDSCommand] (ajp-/127.0.0.1:8702-7) [32126abb] FINISH, DoesImageExistVDSCommand, return: true, log id: 256c0e33 >2014-10-21 15:20:21,513 INFO [org.ovirt.engine.core.bll.ImportVmCommand] (org.ovirt.thread.pool-7-thread-16) [32126abb] Running command: ImportVmCommand internal: false. Entities affected : ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: StorageAction group IMPORT_EXPORT_VM with role type ADMIN >2014-10-21 15:20:21,585 INFO [org.ovirt.engine.core.bll.CopyImageGroupCommand] (org.ovirt.thread.pool-7-thread-16) [5b0d3eed] Running command: CopyImageGroupCommand internal: true. Entities affected : ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: Storage >2014-10-21 15:20:21,653 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (org.ovirt.thread.pool-7-thread-16) [5b0d3eed] START, CopyImageVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = 76b88f40-ff55-49c3-921c-64e0a4c14418, imageGroupId = cae73011-e89d-434b-91b6-c79e60e3df42, imageId = 51aa1ba0-792e-4a6e-8ec5-29e6f1d68d3a, dstImageGroupId = cae73011-e89d-434b-91b6-c79e60e3df42, vmId = ce03198a-a0c4-4745-b442-cf2c5eaeea45, dstImageId = 51aa1ba0-792e-4a6e-8ec5-29e6f1d68d3a, imageDescription = , dstStorageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, copyVolumeType = LeafVol, volumeFormat = RAW, preallocate = Sparse, postZero = false, force = true), log id: 516f7aa8 >2014-10-21 15:20:21,653 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (org.ovirt.thread.pool-7-thread-16) [5b0d3eed] -- executeIrsBrokerCommand: calling 'copyImage' with two new parameters: description and UUID >2014-10-21 15:20:21,653 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (org.ovirt.thread.pool-7-thread-16) [5b0d3eed] -- copyImage parameters: > sdUUID=76b88f40-ff55-49c3-921c-64e0a4c14418 > spUUID=f8f5eaee-8fd0-4b45-87db-62d61b03a916 > vmGUID=ce03198a-a0c4-4745-b442-cf2c5eaeea45 > srcImageGUID=cae73011-e89d-434b-91b6-c79e60e3df42 > srcVolUUID=51aa1ba0-792e-4a6e-8ec5-29e6f1d68d3a > dstImageGUID=cae73011-e89d-434b-91b6-c79e60e3df42 > dstVolUUID=51aa1ba0-792e-4a6e-8ec5-29e6f1d68d3a > descr= > dstSdUUID=a1341057-9cf8-4750-a734-656d05a5d2c5 >2014-10-21 15:20:21,852 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (org.ovirt.thread.pool-7-thread-16) [5b0d3eed] FINISH, CopyImageVDSCommand, return: 00000000-0000-0000-0000-000000000000, log id: 516f7aa8 >2014-10-21 15:20:21,858 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-16) [5b0d3eed] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 25f31ac1-9635-4b30-b0db-c0d88f1aad2b >2014-10-21 15:20:21,858 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-7-thread-16) [5b0d3eed] CommandMultiAsyncTasks::AttachTask: Attaching task 7fefe96d-2c48-4d18-8996-af7d2cdf6752 to command 25f31ac1-9635-4b30-b0db-c0d88f1aad2b. >2014-10-21 15:20:21,873 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-7-thread-16) [5b0d3eed] Adding task 7fefe96d-2c48-4d18-8996-af7d2cdf6752 (Parent Command ImportVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-10-21 15:20:21,984 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-16) [5b0d3eed] START, SetVmStatusVDSCommand( vmId = ce03198a-a0c4-4745-b442-cf2c5eaeea45, status = ImageLocked, exit status = Normal), log id: 5a4da601 >2014-10-21 15:20:21,988 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-16) [5b0d3eed] FINISH, SetVmStatusVDSCommand, log id: 5a4da601 >2014-10-21 15:20:22,053 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-16) [5b0d3eed] Correlation ID: 32126abb, Job ID: 031d85a1-41fa-429f-9c5a-d5db88d876f1, Call Stack: null, Custom Event ID: -1, Message: Starting to import Vm windows7 to Data Center dc_35, Cluster cl_35 >2014-10-21 15:20:22,053 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-16) [5b0d3eed] BaseAsyncTask::startPollingTask: Starting to poll task 7fefe96d-2c48-4d18-8996-af7d2cdf6752. >2014-10-21 15:20:22,059 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-95) Polling and updating Async Tasks: 1 tasks, 1 tasks to poll now >2014-10-21 15:20:22,089 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-95) SPMAsyncTask::PollTask: Polling task 7fefe96d-2c48-4d18-8996-af7d2cdf6752 (Parent Command ImportVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status running. >2014-10-21 15:20:22,089 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-95) Finished polling Tasks, will poll again in 10 seconds. >2014-10-21 15:30:11,806 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-24) SPMAsyncTask::PollTask: Polling task 7fefe96d-2c48-4d18-8996-af7d2cdf6752 (Parent Command ImportVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-10-21 15:30:11,812 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-24) BaseAsyncTask::onTaskEndSuccess: Task 7fefe96d-2c48-4d18-8996-af7d2cdf6752 (Parent Command ImportVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-10-21 15:30:11,815 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-24) CommandAsyncTask::endActionIfNecessary: All tasks of command 25f31ac1-9635-4b30-b0db-c0d88f1aad2b has ended -> executing endAction >2014-10-21 15:30:11,815 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-24) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: 25f31ac1-9635-4b30-b0db-c0d88f1aad2b): calling endAction . >2014-10-21 15:30:11,816 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-5) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction ImportVm, executionIndex: 0 >2014-10-21 15:30:11,836 INFO [org.ovirt.engine.core.bll.ImportVmCommand] (org.ovirt.thread.pool-7-thread-5) Ending command successfully: org.ovirt.engine.core.bll.ImportVmCommand >2014-10-21 15:30:11,857 INFO [org.ovirt.engine.core.bll.CopyImageGroupCommand] (org.ovirt.thread.pool-7-thread-5) [5b0d3eed] Ending command successfully: org.ovirt.engine.core.bll.CopyImageGroupCommand >2014-10-21 15:30:11,871 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-5) [5b0d3eed] START, GetImageInfoVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = cae73011-e89d-434b-91b6-c79e60e3df42, imageId = 51aa1ba0-792e-4a6e-8ec5-29e6f1d68d3a), log id: 268bb780 >2014-10-21 15:30:11,906 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-5) [5b0d3eed] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@d055c47b, log id: 268bb780 >2014-10-21 15:30:11,971 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-5) START, SetVmStatusVDSCommand( vmId = ce03198a-a0c4-4745-b442-cf2c5eaeea45, status = Down, exit status = Normal), log id: 48bf4a8c >2014-10-21 15:30:11,975 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-5) FINISH, SetVmStatusVDSCommand, log id: 48bf4a8c >2014-10-21 15:30:11,975 INFO [org.ovirt.engine.core.bll.ImportVmCommand] (org.ovirt.thread.pool-7-thread-5) Lock freed to object EngineLock [exclusiveLocks= key: windows7 value: VM_NAME >, sharedLocks= key: ce03198a-a0c4-4745-b442-cf2c5eaeea45 value: REMOTE_VM >] >2014-10-21 15:30:11,984 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-5) Correlation ID: 32126abb, Job ID: 031d85a1-41fa-429f-9c5a-d5db88d876f1, Call Stack: null, Custom Event ID: -1, Message: Vm windows7 was imported successfully to Data Center dc_35, Cluster cl_35 >2014-10-21 15:30:11,984 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-5) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type ImportVm completed, handling the result. >2014-10-21 15:30:11,984 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-5) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type ImportVm succeeded, clearing tasks. >2014-10-21 15:30:11,987 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-5) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 7fefe96d-2c48-4d18-8996-af7d2cdf6752 >2014-10-21 15:30:11,988 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-5) START, SPMClearTaskVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, taskId = 7fefe96d-2c48-4d18-8996-af7d2cdf6752), log id: 15f55fd8 >2014-10-21 15:30:11,989 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-5) START, HSMClearTaskVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, taskId=7fefe96d-2c48-4d18-8996-af7d2cdf6752), log id: 2067692b >2014-10-21 15:30:12,027 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-5) FINISH, HSMClearTaskVDSCommand, log id: 2067692b >2014-10-21 15:30:12,027 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-5) FINISH, SPMClearTaskVDSCommand, log id: 15f55fd8 >2014-10-21 15:30:12,033 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-5) BaseAsyncTask::removeTaskFromDB: Removed task 7fefe96d-2c48-4d18-8996-af7d2cdf6752 from DataBase >2014-10-21 15:30:12,033 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-5) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 25f31ac1-9635-4b30-b0db-c0d88f1aad2b >2014-10-21 15:31:38,855 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-46) Setting new tasks map. The map contains now 0 tasks >2014-10-21 15:31:38,855 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-46) Cleared all tasks of pool f8f5eaee-8fd0-4b45-87db-62d61b03a916. >2014-10-21 15:32:26,892 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp-/127.0.0.1:8702-7) [51378fe6] Lock Acquired to object EngineLock [exclusiveLocks= key: ce03198a-a0c4-4745-b442-cf2c5eaeea45 value: VM >, sharedLocks= ] >2014-10-21 15:32:26,985 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp-/127.0.0.1:8702-7) [51378fe6] START, IsVmDuringInitiatingVDSCommand( vmId = ce03198a-a0c4-4745-b442-cf2c5eaeea45), log id: 555875cd >2014-10-21 15:32:26,986 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp-/127.0.0.1:8702-7) [51378fe6] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 555875cd >2014-10-21 15:32:27,019 WARN [org.ovirt.engine.core.bll.RunVmCommand] (ajp-/127.0.0.1:8702-7) [51378fe6] CanDoAction of action RunVm failed. Reasons:VAR__ACTION__RUN,VAR__TYPE__VM,SCHEDULING_NO_HOSTS >2014-10-21 15:32:27,020 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp-/127.0.0.1:8702-7) [51378fe6] Lock freed to object EngineLock [exclusiveLocks= key: ce03198a-a0c4-4745-b442-cf2c5eaeea45 value: VM >, sharedLocks= ] >2014-10-21 15:32:45,854 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (ajp-/127.0.0.1:8702-8) [1dbcd90b] Lock Acquired to object EngineLock [exclusiveLocks= key: ce03198a-a0c4-4745-b442-cf2c5eaeea45 value: VM >, sharedLocks= ] >2014-10-21 15:32:45,913 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-2) [1dbcd90b] Running command: RemoveVmCommand internal: false. Entities affected : ID: ce03198a-a0c4-4745-b442-cf2c5eaeea45 Type: VMAction group DELETE_VM with role type USER >2014-10-21 15:32:45,913 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-2) [1dbcd90b] START, SetVmStatusVDSCommand( vmId = ce03198a-a0c4-4745-b442-cf2c5eaeea45, status = ImageLocked, exit status = Normal), log id: 106c674c >2014-10-21 15:32:45,918 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-2) [1dbcd90b] FINISH, SetVmStatusVDSCommand, log id: 106c674c >2014-10-21 15:32:45,920 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-2) [1dbcd90b] Lock freed to object EngineLock [exclusiveLocks= key: ce03198a-a0c4-4745-b442-cf2c5eaeea45 value: VM >, sharedLocks= ] >2014-10-21 15:32:45,947 INFO [org.ovirt.engine.core.bll.RemoveAllVmImagesCommand] (org.ovirt.thread.pool-7-thread-2) [639e3998] Running command: RemoveAllVmImagesCommand internal: true. Entities affected : ID: ce03198a-a0c4-4745-b442-cf2c5eaeea45 Type: VM >2014-10-21 15:32:45,952 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-7-thread-2) [61aa64d] Running command: RemoveImageCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage >2014-10-21 15:32:45,967 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-7-thread-2) [61aa64d] START, DeleteImageGroupVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = cae73011-e89d-434b-91b6-c79e60e3df42, postZeros = false, forceDelete = false), log id: 38d89aa >2014-10-21 15:32:46,154 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-7-thread-2) [61aa64d] FINISH, DeleteImageGroupVDSCommand, log id: 38d89aa >2014-10-21 15:32:46,159 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-2) [61aa64d] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command bb3aaceb-d794-4613-857a-55fb6e447623 >2014-10-21 15:32:46,160 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-7-thread-2) [61aa64d] CommandMultiAsyncTasks::AttachTask: Attaching task 4bb6acf0-4ab2-45cb-81b3-20f195cbbe68 to command bb3aaceb-d794-4613-857a-55fb6e447623. >2014-10-21 15:32:46,171 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-7-thread-2) [61aa64d] Adding task 4bb6acf0-4ab2-45cb-81b3-20f195cbbe68 (Parent Command RemoveVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-10-21 15:32:46,336 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-2) [61aa64d] Correlation ID: 1dbcd90b, Job ID: 405f45b9-6743-401d-b2c4-e9a3631dbc1c, Call Stack: null, Custom Event ID: -1, Message: VM windows7 was successfully removed. >2014-10-21 15:32:46,336 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-2) [61aa64d] BaseAsyncTask::startPollingTask: Starting to poll task 4bb6acf0-4ab2-45cb-81b3-20f195cbbe68. >2014-10-21 15:32:51,820 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-24) Polling and updating Async Tasks: 1 tasks, 1 tasks to poll now >2014-10-21 15:32:51,849 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-24) SPMAsyncTask::PollTask: Polling task 4bb6acf0-4ab2-45cb-81b3-20f195cbbe68 (Parent Command RemoveVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-10-21 15:32:51,853 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-24) BaseAsyncTask::onTaskEndSuccess: Task 4bb6acf0-4ab2-45cb-81b3-20f195cbbe68 (Parent Command RemoveVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-10-21 15:32:51,854 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-24) CommandAsyncTask::endActionIfNecessary: All tasks of command bb3aaceb-d794-4613-857a-55fb6e447623 has ended -> executing endAction >2014-10-21 15:32:51,854 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-24) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: bb3aaceb-d794-4613-857a-55fb6e447623): calling endAction . >2014-10-21 15:32:51,854 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-15) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction RemoveVm, executionIndex: 0 >2014-10-21 15:32:51,870 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-15) Ending command successfully: org.ovirt.engine.core.bll.RemoveVmCommand >2014-10-21 15:32:51,871 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-15) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RemoveVm completed, handling the result. >2014-10-21 15:32:51,871 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-15) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RemoveVm succeeded, clearing tasks. >2014-10-21 15:32:51,873 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-15) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 4bb6acf0-4ab2-45cb-81b3-20f195cbbe68 >2014-10-21 15:32:51,874 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-15) START, SPMClearTaskVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, taskId = 4bb6acf0-4ab2-45cb-81b3-20f195cbbe68), log id: 2850ad21 >2014-10-21 15:32:51,876 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-15) START, HSMClearTaskVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, taskId=4bb6acf0-4ab2-45cb-81b3-20f195cbbe68), log id: 21c2489a >2014-10-21 15:32:51,904 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-15) FINISH, HSMClearTaskVDSCommand, log id: 21c2489a >2014-10-21 15:32:51,904 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-15) FINISH, SPMClearTaskVDSCommand, log id: 2850ad21 >2014-10-21 15:32:51,909 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-15) BaseAsyncTask::removeTaskFromDB: Removed task 4bb6acf0-4ab2-45cb-81b3-20f195cbbe68 from DataBase >2014-10-21 15:32:51,909 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-15) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity bb3aaceb-d794-4613-857a-55fb6e447623 >2014-10-21 15:33:50,671 INFO [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (ajp-/127.0.0.1:8702-5) [3fc908f0] Lock Acquired to object EngineLock [exclusiveLocks= key: 76b88f40-ff55-49c3-921c-64e0a4c14418 value: STORAGE >, sharedLocks= ] >2014-10-21 15:33:50,765 INFO [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (org.ovirt.thread.pool-7-thread-38) [3fc908f0] Running command: DeactivateStorageDomainCommand internal: false. Entities affected : ID: 76b88f40-ff55-49c3-921c-64e0a4c14418 Type: StorageAction group MANIPULATE_STORAGE_DOMAIN with role type ADMIN >2014-10-21 15:33:50,773 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeactivateStorageDomainVDSCommand] (org.ovirt.thread.pool-7-thread-38) [3fc908f0] START, DeactivateStorageDomainVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = 76b88f40-ff55-49c3-921c-64e0a4c14418, masterDomainId = 00000000-0000-0000-0000-000000000000, masterVersion = 1), log id: 13544cdc >2014-10-21 15:33:50,799 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeactivateStorageDomainVDSCommand] (org.ovirt.thread.pool-7-thread-38) [3fc908f0] FINISH, DeactivateStorageDomainVDSCommand, log id: 13544cdc >2014-10-21 15:33:50,799 INFO [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (org.ovirt.thread.pool-7-thread-38) [3fc908f0] Lock freed to object EngineLock [exclusiveLocks= key: 76b88f40-ff55-49c3-921c-64e0a4c14418 value: STORAGE >, sharedLocks= ] >2014-10-21 15:33:50,811 INFO [org.ovirt.engine.core.bll.storage.AfterDeactivateSingleAsyncOperation] (org.ovirt.thread.pool-7-thread-20) After deactivate treatment vds: alma06.qa.lab.tlv.redhat.com,pool dc_35 >2014-10-21 15:33:50,815 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-7-thread-20) START, ConnectStoragePoolVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vdsId = 1c39bd88-8672-408f-a593-60d6ef8f647c, storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, masterVersion = 1), log id: 299b4f74 >2014-10-21 15:33:50,818 INFO [org.ovirt.engine.core.bll.storage.AfterDeactivateSingleAsyncOperation] (org.ovirt.thread.pool-7-thread-25) After deactivate treatment vds: alma07.qa.lab.tlv.redhat.com,pool dc_35 >2014-10-21 15:33:50,823 INFO [org.ovirt.engine.core.bll.storage.AfterDeactivateSingleAsyncOperation] (org.ovirt.thread.pool-7-thread-12) After deactivate treatment vds: cyan-vdsg.qa.lab.tlv.redhat.com,pool dc_35 >2014-10-21 15:33:50,823 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-7-thread-25) START, ConnectStoragePoolVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vdsId = d5b2492d-8110-4614-9ce4-4a7a118c7942, storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, masterVersion = 1), log id: 6e749d67 >2014-10-21 15:33:50,825 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-7-thread-12) START, ConnectStoragePoolVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, vdsId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, masterVersion = 1), log id: 25be1b29 >2014-10-21 15:33:51,144 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-7-thread-25) FINISH, ConnectStoragePoolVDSCommand, log id: 6e749d67 >2014-10-21 15:33:51,148 INFO [org.ovirt.engine.core.bll.storage.DisconnectStorageServerConnectionCommand] (org.ovirt.thread.pool-7-thread-25) [4f43525d] Running command: DisconnectStorageServerConnectionCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-10-21 15:33:51,150 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-25) [4f43525d] START, DisconnectStorageServerVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: a4d5f882-3b1b-43ad-9ccb-cca623ff7d1b, connection: 10.35.160.108:/RHEV/mom_export, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 20dfc9ed >2014-10-21 15:33:51,155 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-7-thread-20) FINISH, ConnectStoragePoolVDSCommand, log id: 299b4f74 >2014-10-21 15:33:51,158 INFO [org.ovirt.engine.core.bll.storage.DisconnectStorageServerConnectionCommand] (org.ovirt.thread.pool-7-thread-20) [8059dd9] Running command: DisconnectStorageServerConnectionCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-10-21 15:33:51,167 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-20) [8059dd9] START, DisconnectStorageServerVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: a4d5f882-3b1b-43ad-9ccb-cca623ff7d1b, connection: 10.35.160.108:/RHEV/mom_export, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 4eff5230 >2014-10-21 15:33:51,291 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-25) [4f43525d] FINISH, DisconnectStorageServerVDSCommand, return: {a4d5f882-3b1b-43ad-9ccb-cca623ff7d1b=0}, log id: 20dfc9ed >2014-10-21 15:33:51,328 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-20) [8059dd9] FINISH, DisconnectStorageServerVDSCommand, return: {a4d5f882-3b1b-43ad-9ccb-cca623ff7d1b=0}, log id: 4eff5230 >2014-10-21 15:33:52,394 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-7-thread-12) FINISH, ConnectStoragePoolVDSCommand, log id: 25be1b29 >2014-10-21 15:33:52,398 INFO [org.ovirt.engine.core.bll.storage.DisconnectStorageServerConnectionCommand] (org.ovirt.thread.pool-7-thread-38) [37f55fd9] Running command: DisconnectStorageServerConnectionCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-10-21 15:33:52,403 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-38) [37f55fd9] START, DisconnectStorageServerVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: a4d5f882-3b1b-43ad-9ccb-cca623ff7d1b, connection: 10.35.160.108:/RHEV/mom_export, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 7c78b944 >2014-10-21 15:33:53,377 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-7-thread-2) Adding domain 76b88f40-ff55-49c3-921c-64e0a4c14418 to the domains in maintenance cache >2014-10-21 15:33:53,557 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-38) [37f55fd9] FINISH, DisconnectStorageServerVDSCommand, return: {a4d5f882-3b1b-43ad-9ccb-cca623ff7d1b=0}, log id: 7c78b944 >2014-10-21 15:33:53,558 INFO [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (org.ovirt.thread.pool-7-thread-38) [37f55fd9] Domain 76b88f40-ff55-49c3-921c-64e0a4c14418 will remain in PreparingForMaintenance status until deactivated on all hosts >2014-10-21 15:33:53,569 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-38) [37f55fd9] Correlation ID: 3fc908f0, Job ID: ea88b3ff-7c52-4811-8ff6-0673bd4fb602, Call Stack: null, Custom Event ID: -1, Message: Storage Domain mom_export (Data Center dc_35) was deactivated by admin >2014-10-21 15:34:04,104 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-7-thread-23) Moving domain 76b88f40-ff55-49c3-921c-64e0a4c14418 to maintenance >2014-10-21 15:34:08,858 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-59) Setting new tasks map. The map contains now 0 tasks >2014-10-21 15:34:08,858 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-59) Cleared all tasks of pool f8f5eaee-8fd0-4b45-87db-62d61b03a916. >2014-10-21 15:58:20,659 INFO [org.ovirt.engine.core.bll.storage.DetachStorageDomainFromPoolCommand] (ajp-/127.0.0.1:8702-7) [7edb8bd9] Lock Acquired to object EngineLock [exclusiveLocks= key: 76b88f40-ff55-49c3-921c-64e0a4c14418 value: STORAGE >, sharedLocks= ] >2014-10-21 15:58:20,711 INFO [org.ovirt.engine.core.bll.storage.DetachStorageDomainFromPoolCommand] (org.ovirt.thread.pool-7-thread-12) [7edb8bd9] Running command: DetachStorageDomainFromPoolCommand internal: false. Entities affected : ID: 76b88f40-ff55-49c3-921c-64e0a4c14418 Type: StorageAction group MANIPULATE_STORAGE_DOMAIN with role type ADMIN >2014-10-21 15:58:20,712 INFO [org.ovirt.engine.core.bll.storage.DetachStorageDomainFromPoolCommand] (org.ovirt.thread.pool-7-thread-12) [7edb8bd9] Start detach storage domain >2014-10-21 15:58:20,715 INFO [org.ovirt.engine.core.bll.storage.DetachStorageDomainFromPoolCommand] (org.ovirt.thread.pool-7-thread-12) [7edb8bd9] Detach storage domain: before connect >2014-10-21 15:58:20,721 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (org.ovirt.thread.pool-7-thread-30) [7b0b560d] Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-10-21 15:58:20,722 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (org.ovirt.thread.pool-7-thread-18) [253eae97] Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-10-21 15:58:20,725 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-30) [7b0b560d] START, ConnectStorageServerVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: a4d5f882-3b1b-43ad-9ccb-cca623ff7d1b, connection: 10.35.160.108:/RHEV/mom_export, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 5b20df4b >2014-10-21 15:58:20,726 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (org.ovirt.thread.pool-7-thread-47) [23e91dd6] Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-10-21 15:58:20,730 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-47) [23e91dd6] START, ConnectStorageServerVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: a4d5f882-3b1b-43ad-9ccb-cca623ff7d1b, connection: 10.35.160.108:/RHEV/mom_export, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 1cceb3f >2014-10-21 15:58:20,732 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-18) [253eae97] START, ConnectStorageServerVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: a4d5f882-3b1b-43ad-9ccb-cca623ff7d1b, connection: 10.35.160.108:/RHEV/mom_export, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 2137d470 >2014-10-21 15:58:20,820 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-47) [23e91dd6] FINISH, ConnectStorageServerVDSCommand, return: {a4d5f882-3b1b-43ad-9ccb-cca623ff7d1b=0}, log id: 1cceb3f >2014-10-21 15:58:20,838 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-18) [253eae97] FINISH, ConnectStorageServerVDSCommand, return: {a4d5f882-3b1b-43ad-9ccb-cca623ff7d1b=0}, log id: 2137d470 >2014-10-21 15:58:20,964 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-30) [7b0b560d] FINISH, ConnectStorageServerVDSCommand, return: {a4d5f882-3b1b-43ad-9ccb-cca623ff7d1b=0}, log id: 5b20df4b >2014-10-21 15:58:20,966 INFO [org.ovirt.engine.core.bll.storage.DetachStorageDomainFromPoolCommand] (org.ovirt.thread.pool-7-thread-12) [7edb8bd9] Detach storage domain: after connect >2014-10-21 15:58:20,968 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DetachStorageDomainVDSCommand] (org.ovirt.thread.pool-7-thread-12) [7edb8bd9] START, DetachStorageDomainVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = 76b88f40-ff55-49c3-921c-64e0a4c14418, masterDomainId = 00000000-0000-0000-0000-000000000000, masterVersion = 1, force = false), log id: 2678006a >2014-10-21 15:58:21,271 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DetachStorageDomainVDSCommand] (org.ovirt.thread.pool-7-thread-12) [7edb8bd9] FINISH, DetachStorageDomainVDSCommand, log id: 2678006a >2014-10-21 15:58:21,271 INFO [org.ovirt.engine.core.bll.storage.DetachStorageDomainFromPoolCommand] (org.ovirt.thread.pool-7-thread-12) [7edb8bd9] Detach storage domain: after detach in vds >2014-10-21 15:58:21,281 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-7-thread-25) START, ConnectStoragePoolVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vdsId = d5b2492d-8110-4614-9ce4-4a7a118c7942, storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, masterVersion = 1), log id: 555057f2 >2014-10-21 15:58:21,284 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-7-thread-2) START, ConnectStoragePoolVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, vdsId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, masterVersion = 1), log id: 2a336799 >2014-10-21 15:58:21,285 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-7-thread-38) START, ConnectStoragePoolVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vdsId = 1c39bd88-8672-408f-a593-60d6ef8f647c, storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, masterVersion = 1), log id: 3ccf850d >2014-10-21 15:58:21,588 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-7-thread-25) FINISH, ConnectStoragePoolVDSCommand, log id: 555057f2 >2014-10-21 15:58:21,592 INFO [org.ovirt.engine.core.bll.storage.DisconnectStorageServerConnectionCommand] (org.ovirt.thread.pool-7-thread-25) [5502d72c] Running command: DisconnectStorageServerConnectionCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-10-21 15:58:21,592 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-7-thread-38) FINISH, ConnectStoragePoolVDSCommand, log id: 3ccf850d >2014-10-21 15:58:21,595 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-25) [5502d72c] START, DisconnectStorageServerVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: a4d5f882-3b1b-43ad-9ccb-cca623ff7d1b, connection: 10.35.160.108:/RHEV/mom_export, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 56632b36 >2014-10-21 15:58:21,596 INFO [org.ovirt.engine.core.bll.storage.DisconnectStorageServerConnectionCommand] (org.ovirt.thread.pool-7-thread-38) [4318bee1] Running command: DisconnectStorageServerConnectionCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-10-21 15:58:21,597 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-38) [4318bee1] START, DisconnectStorageServerVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: a4d5f882-3b1b-43ad-9ccb-cca623ff7d1b, connection: 10.35.160.108:/RHEV/mom_export, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 5a4eeea2 >2014-10-21 15:58:21,715 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-25) [5502d72c] FINISH, DisconnectStorageServerVDSCommand, return: {a4d5f882-3b1b-43ad-9ccb-cca623ff7d1b=0}, log id: 56632b36 >2014-10-21 15:58:21,737 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-38) [4318bee1] FINISH, DisconnectStorageServerVDSCommand, return: {a4d5f882-3b1b-43ad-9ccb-cca623ff7d1b=0}, log id: 5a4eeea2 >2014-10-21 15:58:21,916 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-7-thread-2) FINISH, ConnectStoragePoolVDSCommand, log id: 2a336799 >2014-10-21 15:58:21,919 INFO [org.ovirt.engine.core.bll.storage.DisconnectStorageServerConnectionCommand] (org.ovirt.thread.pool-7-thread-2) [1dc3a64d] Running command: DisconnectStorageServerConnectionCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-10-21 15:58:21,920 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-2) [1dc3a64d] START, DisconnectStorageServerVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: a4d5f882-3b1b-43ad-9ccb-cca623ff7d1b, connection: 10.35.160.108:/RHEV/mom_export, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 31254615 >2014-10-21 15:58:22,147 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-2) [1dc3a64d] FINISH, DisconnectStorageServerVDSCommand, return: {a4d5f882-3b1b-43ad-9ccb-cca623ff7d1b=0}, log id: 31254615 >2014-10-21 15:58:22,148 INFO [org.ovirt.engine.core.bll.storage.DetachStorageDomainFromPoolCommand] (org.ovirt.thread.pool-7-thread-12) [7edb8bd9] Detach storage domain: after disconnect storage >2014-10-21 15:58:22,261 INFO [org.ovirt.engine.core.bll.storage.DetachStorageDomainFromPoolCommand] (org.ovirt.thread.pool-7-thread-12) [7edb8bd9] End detach storage domain >2014-10-21 15:58:22,268 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-12) [7edb8bd9] Correlation ID: 7edb8bd9, Job ID: a3459f75-e1ad-40ab-a57e-11895e5c1df2, Call Stack: null, Custom Event ID: -1, Message: Storage Domain mom_export was detached from Data Center dc_35 by admin >2014-10-21 15:58:22,271 INFO [org.ovirt.engine.core.bll.storage.DetachStorageDomainFromPoolCommand] (org.ovirt.thread.pool-7-thread-12) [7edb8bd9] Lock freed to object EngineLock [exclusiveLocks= key: 76b88f40-ff55-49c3-921c-64e0a4c14418 value: STORAGE >, sharedLocks= ] >2014-10-21 15:58:30,204 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-7-thread-23) Removing domain 76b88f40-ff55-49c3-921c-64e0a4c14418 from the domains in maintenance cache >2014-10-21 15:59:50,504 INFO [org.ovirt.engine.core.bll.storage.RemoveStorageDomainCommand] (ajp-/127.0.0.1:8702-11) [7b91e337] Lock Acquired to object EngineLock [exclusiveLocks= key: 76b88f40-ff55-49c3-921c-64e0a4c14418 value: STORAGE >, sharedLocks= ] >2014-10-21 15:59:50,515 INFO [org.ovirt.engine.core.bll.storage.RemoveStorageDomainCommand] (ajp-/127.0.0.1:8702-11) [7b91e337] Running command: RemoveStorageDomainCommand internal: false. Entities affected : ID: 76b88f40-ff55-49c3-921c-64e0a4c14418 Type: StorageAction group DELETE_STORAGE_DOMAIN with role type ADMIN >2014-10-21 15:59:50,615 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) [7b91e337] Correlation ID: 7b91e337, Job ID: 005ba75f-22c2-447b-aca2-182ccad4c2de, Call Stack: null, Custom Event ID: -1, Message: Storage Domain mom_export was removed by admin >2014-10-21 15:59:50,619 INFO [org.ovirt.engine.core.bll.storage.RemoveStorageDomainCommand] (ajp-/127.0.0.1:8702-11) [7b91e337] Lock freed to object EngineLock [exclusiveLocks= key: 76b88f40-ff55-49c3-921c-64e0a4c14418 value: STORAGE >, sharedLocks= ] >2014-10-21 16:03:43,778 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-40) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 16:04:07,586 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-57) Deactivating non existing principal admin::internal >2014-10-21 16:04:14,118 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-46) Attempting to update VMs/Templates Ovf. >2014-10-21 16:07:06,900 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-85) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 16:07:43,470 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-30) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 17:03:57,564 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-87) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 17:04:14,150 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-14) Attempting to update VMs/Templates Ovf. >2014-10-21 17:07:07,188 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-73) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 17:07:49,572 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-83) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 18:04:12,116 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-36) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 18:04:14,167 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-69) Attempting to update VMs/Templates Ovf. >2014-10-21 18:07:09,836 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) [1883f47f] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 18:07:59,586 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-73) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 19:04:14,199 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-38) Attempting to update VMs/Templates Ovf. >2014-10-21 19:04:26,188 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) [1883f47f] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 19:07:24,777 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-9) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 19:08:06,582 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-37) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 20:04:14,231 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-26) Attempting to update VMs/Templates Ovf. >2014-10-21 20:04:39,942 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-50) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 20:07:39,977 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-83) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 20:08:13,323 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-34) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 21:04:14,247 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-89) Attempting to update VMs/Templates Ovf. >2014-10-21 21:04:53,983 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-44) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 21:07:54,496 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-15) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 21:08:20,300 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-81) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 22:04:14,279 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-68) Attempting to update VMs/Templates Ovf. >2014-10-21 22:05:07,193 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-92) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 22:08:09,215 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-78) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 22:08:30,106 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-76) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 23:04:14,309 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-13) Attempting to update VMs/Templates Ovf. >2014-10-21 23:05:20,628 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-76) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 23:08:23,934 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-67) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-21 23:08:37,250 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-56) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 00:04:14,338 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-44) Attempting to update VMs/Templates Ovf. >2014-10-22 00:05:33,640 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 00:08:38,912 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-10) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 00:08:46,067 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-67) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 01:04:14,355 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-12) Attempting to update VMs/Templates Ovf. >2014-10-22 01:05:47,750 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-57) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 01:08:53,576 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-19) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 01:08:54,134 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-59) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 02:04:14,372 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-72) Attempting to update VMs/Templates Ovf. >2014-10-22 02:06:01,341 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-72) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 02:09:03,156 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-70) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 02:09:08,457 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-21) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 03:04:14,402 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-88) Attempting to update VMs/Templates Ovf. >2014-10-22 03:06:14,803 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-30) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 03:09:10,588 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-45) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 03:09:23,234 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-69) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 03:35:35,001 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-5) Start deleteAgedOutAuditLogs >2014-10-22 03:35:35,002 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-95) Start deleteAgedOutCommandEntities >2014-10-22 03:35:35,007 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-95) Finished deleteAgedOutCommandEntities >2014-10-22 03:35:35,008 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-5) Finished deleteAgedOutAuditLogs >2014-10-22 04:04:14,418 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-95) Attempting to update VMs/Templates Ovf. >2014-10-22 04:06:27,875 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-50) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 04:09:18,081 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-30) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 04:09:38,324 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-81) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 05:04:14,443 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-95) Attempting to update VMs/Templates Ovf. >2014-10-22 05:06:40,462 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-70) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 05:09:25,070 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-17) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 05:09:52,829 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-17) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 06:04:14,459 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-73) Attempting to update VMs/Templates Ovf. >2014-10-22 06:06:55,520 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-44) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 06:09:33,804 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-99) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 06:10:07,616 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-77) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 07:04:14,476 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-44) Attempting to update VMs/Templates Ovf. >2014-10-22 07:07:08,905 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-45) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 07:09:41,010 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-35) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 418 seconds while maximum configured value is 300 seconds. >2014-10-22 07:10:07,897 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-94) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 08:04:14,493 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-38) Attempting to update VMs/Templates Ovf. >2014-10-22 08:07:22,354 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-79) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 08:09:48,887 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-8) [5c27910a] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 08:10:22,862 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 09:04:14,510 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-79) Attempting to update VMs/Templates Ovf. >2014-10-22 09:07:35,370 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-19) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 09:09:55,535 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-55) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 09:10:37,622 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-14) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 10:04:14,526 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-17) Attempting to update VMs/Templates Ovf. >2014-10-22 10:07:48,544 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-2) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 10:10:02,269 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-24) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 10:10:52,416 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-86) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 11:04:14,541 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-88) Attempting to update VMs/Templates Ovf. >2014-10-22 11:08:03,274 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-65) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 11:10:09,108 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-29) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 11:10:53,596 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 12:04:14,558 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-48) Attempting to update VMs/Templates Ovf. >2014-10-22 12:05:08,291 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-2) Running command: LoginUserCommand internal: false. >2014-10-22 12:05:08,295 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-10-22 12:05:21,087 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-11) [53cc0416] Running command: LogoutUserCommand internal: false. >2014-10-22 12:05:21,090 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-1) [789fafe2] Running command: LogoutUserCommand internal: false. >2014-10-22 12:05:21,091 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-3) [72f7a043] Running command: LogoutUserCommand internal: false. >2014-10-22 12:05:21,093 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-12) [24470156] Running command: LogoutUserCommand internal: false. >2014-10-22 12:05:21,095 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [789fafe2] Correlation ID: 789fafe2, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-10-22 12:05:21,097 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) [72f7a043] Correlation ID: 72f7a043, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-10-22 12:05:21,097 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) [53cc0416] Correlation ID: 53cc0416, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-10-22 12:05:21,099 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-12) [24470156] Correlation ID: 24470156, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-10-22 12:05:21,123 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-7) Invalid locale found in configuration: >2014-10-22 12:05:21,123 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-7) Invalid locale found in configuration: >2014-10-22 12:08:17,082 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-12) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 12:10:18,393 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-24) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 12:10:53,663 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-18) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 13:04:07,648 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-89) Deactivating non existing principal admin::internal >2014-10-22 13:04:14,573 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-64) Attempting to update VMs/Templates Ovf. >2014-10-22 13:08:30,803 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-43) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 13:10:27,193 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-74) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 13:10:54,537 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-99) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 13:39:34,536 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcherImpl.write0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47) [rt.jar:1.7.0_65] > at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93) [rt.jar:1.7.0_65] > at sun.nio.ch.IOUtil.write(IOUtil.java:65) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:487) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:95) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.write(SSLClient.java:90) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.processOutgoing(ReactorClient.java:199) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.process(ReactorClient.java:170) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:115) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] > >2014-10-22 13:42:33,974 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ListVDSCommand] (DefaultQuartzScheduler_Worker-58) Command ListVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 13:42:33,975 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-58) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 60 seconds and after that an attempt to fence the host will be issued. >2014-10-22 13:42:33,982 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-58) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 60 seconds and after that an attempt to fence the host will be issued. >2014-10-22 13:42:33,982 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-58) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.ListVDSCommand.executeVdsBrokerCommand(ListVDSCommand.java:24) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:418) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.fetchRunningVms(VdsUpdateRunTimeInfo.java:1000) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVmStats(VdsUpdateRunTimeInfo.java:949) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:667) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 13:42:33,990 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-58) Failed to refresh VDS , vds = d5b2492d-8110-4614-9ce4-4a7a118c7942 : alma07.qa.lab.tlv.redhat.com, VDS Network Error, continuing. >VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 13:42:37,002 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 13:42:37,052 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 13:45:39,040 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-67) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@64fab3e7 >2014-10-22 13:45:39,041 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-67) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 13:45:39,041 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-67) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 13:45:39,041 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-67) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 60 seconds and after that an attempt to fence the host will be issued. >2014-10-22 13:45:39,049 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-67) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 60 seconds and after that an attempt to fence the host will be issued. >2014-10-22 13:45:39,050 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-67) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 13:45:39,055 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-67) Failed to refresh VDS , vds = d5b2492d-8110-4614-9ce4-4a7a118c7942 : alma07.qa.lab.tlv.redhat.com, VDS Network Error, continuing. >VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 13:45:42,579 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 13:45:42,590 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 13:48:44,604 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-53) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@690a1af0 >2014-10-22 13:48:44,605 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-53) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 13:48:44,606 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-53) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 13:48:44,606 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-53) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 60 seconds and after that an attempt to fence the host will be issued. >2014-10-22 13:48:44,615 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-53) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 60 seconds and after that an attempt to fence the host will be issued. >2014-10-22 13:48:44,685 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-53) Server failed to respond, vds_id = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds_name = alma07.qa.lab.tlv.redhat.com, vm_count = 0, spm_status = None, non-responsive_timeout (seconds) = 60, error = VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 13:48:44,691 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-53) Correlation ID: null, Call Stack: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) >, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is non responsive. >2014-10-22 13:48:44,695 INFO [org.ovirt.engine.core.bll.VdsEventListener] (org.ovirt.thread.pool-7-thread-47) ResourceManager::vdsNotResponding entered for Host d5b2492d-8110-4614-9ce4-4a7a118c7942, alma07.qa.lab.tlv.redhat.com >2014-10-22 13:48:44,705 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-53) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 13:48:44,721 INFO [org.ovirt.engine.core.bll.SshSoftFencingCommand] (org.ovirt.thread.pool-7-thread-47) [4ed3b2c2] Running command: SshSoftFencingCommand internal: true. Entities affected : ID: d5b2492d-8110-4614-9ce4-4a7a118c7942 Type: VDS >2014-10-22 13:48:44,728 INFO [org.ovirt.engine.core.bll.SshSoftFencingCommand] (org.ovirt.thread.pool-7-thread-47) [4ed3b2c2] Opening SSH Soft Fencing session on host alma07.qa.lab.tlv.redhat.com >2014-10-22 13:48:45,262 ERROR [org.ovirt.engine.core.bll.SshSoftFencingCommand] (org.ovirt.thread.pool-7-thread-47) [4ed3b2c2] SSH Soft Fencing command failed on host alma07.qa.lab.tlv.redhat.com: SSH authentication to 'root@alma07.qa.lab.tlv.redhat.com' failed. Please verify provided credentials. Make sure key is authorized at host >Stdout: >Stderr: >Stacktrace: javax.naming.AuthenticationException: SSH authentication to 'root@alma07.qa.lab.tlv.redhat.com' failed. Please verify provided credentials. Make sure key is authorized at host: javax.naming.AuthenticationException: SSH authentication to 'root@alma07.qa.lab.tlv.redhat.com' failed. Please verify provided credentials. Make sure key is authorized at host > at org.ovirt.engine.core.uutils.ssh.SSHClient.authenticate(SSHClient.java:365) [uutils.jar:] > at org.ovirt.engine.core.bll.SshSoftFencingCommand.executeSshSoftFencingCommand(SshSoftFencingCommand.java:88) [bll.jar:] > at org.ovirt.engine.core.bll.SshSoftFencingCommand.executeCommand(SshSoftFencingCommand.java:53) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1168) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1307) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1932) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1331) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:344) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:430) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:411) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:616) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor290.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52) [jboss-as-ee.jar:7.4.1.Final-redhat-3] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) [jboss-as-ee.jar:7.4.1.Final-redhat-3] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:74) [jboss-as-weld.jar:7.4.1.Final-redhat-3] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:84) [jboss-as-weld.jar:7.4.1.Final-redhat-3] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:97) [jboss-as-weld.jar:7.4.1.Final-redhat-3] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) [jboss-as-ee.jar:7.4.1.Final-redhat-3] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) [jboss-as-ee.jar:7.4.1.Final-redhat-3] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [jboss-as-ejb3.jar:7.4.1.Final-redhat-3] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:93) [jboss-as-weld.jar:7.4.1.Final-redhat-3] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee.jar:7.4.1.Final-redhat-3] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3.jar:7.4.1.Final-redhat-3] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:259) [jboss-as-ejb3.jar:7.4.1.Final-redhat-3] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:398) [jboss-as-ejb3.jar:7.4.1.Final-redhat-3] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:242) [jboss-as-ejb3.jar:7.4.1.Final-redhat-3] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3.jar:7.4.1.Final-redhat-3] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3.jar:7.4.1.Final-redhat-3] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3.jar:7.4.1.Final-redhat-3] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee.jar:7.4.1.Final-redhat-3] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee.jar:7.4.1.Final-redhat-3] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:185) [jboss-as-ee.jar:7.4.1.Final-redhat-3] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee.jar:7.4.1.Final-redhat-3] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73) [jboss-as-ee.jar:7.4.1.Final-redhat-3] > at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view7.runInternalAction(Unknown Source) [bll.jar:] > at org.ovirt.engine.core.bll.VdsEventListener$4.run(VdsEventListener.java:249) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:90) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_65] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_65] > at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_65] > >2014-10-22 13:48:45,386 INFO [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-7-thread-47) [65f96f40] Lock Acquired to object EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_FENCE >, sharedLocks= ] >2014-10-22 13:48:45,387 ERROR [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-7-thread-47) [65f96f40] Failed to run Fence script on vds:alma07.qa.lab.tlv.redhat.com. >2014-10-22 13:48:45,397 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-47) [65f96f40] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com became non responsive. It has no power management configured. Please check the host status, manually reboot it, and click "Confirm Host Has Been Rebooted" >2014-10-22 13:48:45,397 WARN [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-7-thread-47) [65f96f40] CanDoAction of action VdsNotRespondingTreatment failed. Reasons:VAR__ACTION__RESTART,VDS_FENCE_DISABLED >2014-10-22 13:48:45,397 INFO [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-7-thread-47) [65f96f40] Lock freed to object EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS_FENCE >, sharedLocks= ] >2014-10-22 13:48:47,716 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 13:48:47,726 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 13:51:50,169 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-35) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@7c24c9af >2014-10-22 13:51:50,169 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-35) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 13:51:50,169 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-35) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 13:51:50,170 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-35) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 13:51:50,177 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-35) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 13:51:50,182 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-35) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 13:51:53,323 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 13:51:53,338 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 13:54:55,740 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-95) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@3b9a17a8 >2014-10-22 13:54:55,740 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-95) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 13:54:55,741 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-95) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 13:54:55,741 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-95) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 13:54:55,748 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-95) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 13:54:55,753 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-95) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 13:54:58,762 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 13:54:58,791 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 13:58:00,802 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-75) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@b5c93f6 >2014-10-22 13:58:00,802 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-75) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 13:58:00,802 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-75) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 13:58:00,802 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-75) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 13:58:00,809 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-75) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 13:58:00,815 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-75) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 13:58:04,042 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 13:58:04,055 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 14:01:06,384 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-34) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@7adec3d1 >2014-10-22 14:01:06,384 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-34) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 14:01:06,384 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-34) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 14:01:06,384 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-34) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 14:01:06,392 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-34) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 14:01:06,397 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-34) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 14:01:09,408 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 14:01:09,423 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 14:04:11,459 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-1) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@74faa574 >2014-10-22 14:04:11,460 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-1) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 14:04:11,460 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-1) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 14:04:11,460 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-1) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 14:04:11,467 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-1) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 14:04:11,472 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-1) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 14:04:14,588 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-35) Attempting to update VMs/Templates Ovf. >2014-10-22 14:04:14,612 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 14:04:14,623 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 14:07:17,019 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-17) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@49ae9468 >2014-10-22 14:07:17,020 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-17) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 14:07:17,020 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-17) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 14:07:17,020 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-17) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 14:07:17,028 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-17) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 14:07:17,033 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-17) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 14:07:20,043 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 14:07:20,061 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 14:10:22,077 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-28) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@1306adb6 >2014-10-22 14:10:22,078 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-28) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 14:10:22,078 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-28) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 14:10:22,078 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-28) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 14:10:22,085 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-28) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 14:10:22,090 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-28) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 14:10:25,232 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 14:10:25,257 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 14:10:33,614 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-18) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 14:10:55,737 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-96) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 14:13:27,636 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-52) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@36157c4 >2014-10-22 14:13:27,636 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-52) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 14:13:27,636 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-52) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 14:13:27,637 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-52) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 14:13:27,645 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-52) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 14:13:27,650 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-52) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 14:13:30,659 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 14:13:30,684 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 14:16:32,703 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-39) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@63967f37 >2014-10-22 14:16:32,703 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-39) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 14:16:32,703 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-39) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 14:16:32,703 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-39) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 14:16:32,710 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-39) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 14:16:32,715 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-39) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 14:16:35,939 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 14:16:35,950 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 14:19:38,274 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-2) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@2dc394ca >2014-10-22 14:19:38,275 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-2) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 14:19:38,275 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-2) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 14:19:38,275 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-2) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 14:19:38,282 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-2) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 14:19:38,287 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-2) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 14:19:41,298 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 14:19:41,306 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 14:22:43,332 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-44) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@5d0f4455 >2014-10-22 14:22:43,332 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-44) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 14:22:43,333 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-44) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 14:22:43,334 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-44) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 14:22:43,349 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-44) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 14:22:43,355 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-44) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 14:22:46,749 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 14:22:46,758 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 14:25:48,907 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-94) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@6cd95979 >2014-10-22 14:25:48,907 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-94) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 14:25:48,907 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-94) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 14:25:48,907 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-94) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 14:25:48,915 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-94) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 14:25:48,920 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-94) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 14:25:51,930 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 14:25:51,940 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 14:28:53,962 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-84) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@3173d76f >2014-10-22 14:28:53,962 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-84) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 14:28:53,962 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-84) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 14:28:53,963 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-84) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 14:28:53,972 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-84) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 14:28:53,977 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-84) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 14:28:57,121 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 14:28:57,143 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 14:31:59,540 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-23) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@5883bb63 >2014-10-22 14:31:59,541 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-23) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 14:31:59,541 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-23) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 14:31:59,541 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-23) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 14:31:59,548 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-23) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 14:31:59,554 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-23) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 14:32:02,566 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 14:32:02,573 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 14:35:04,601 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-59) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@23624122 >2014-10-22 14:35:04,601 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-59) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 14:35:04,601 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-59) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 14:35:04,602 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-59) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 14:35:04,609 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-59) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 14:35:04,614 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-59) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 14:35:08,052 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 14:35:08,060 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 14:38:10,161 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-7) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@2a6ea6e7 >2014-10-22 14:38:10,161 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-7) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 14:38:10,162 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-7) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 14:38:10,162 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-7) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 14:38:10,168 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 14:38:10,172 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-7) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 14:38:13,184 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 14:38:13,192 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 14:41:15,229 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-14) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@7726d86b >2014-10-22 14:41:15,229 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-14) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 14:41:15,229 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-14) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 14:41:15,230 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-14) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 14:41:15,244 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-14) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 14:41:15,250 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-14) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 14:41:18,477 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 14:41:18,496 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 14:44:20,805 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-13) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@79c8ffc0 >2014-10-22 14:44:20,806 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-13) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 14:44:20,806 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-13) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 14:44:20,806 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-13) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 14:44:20,813 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-13) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 14:44:20,818 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-13) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 14:44:23,827 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 14:44:23,834 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 14:47:25,870 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-92) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@5a9dbc23 >2014-10-22 14:47:25,870 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-92) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 14:47:25,871 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-92) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 14:47:25,871 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-92) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 14:47:25,886 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-92) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 14:47:25,890 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-92) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 14:47:29,294 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 14:47:29,301 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 14:50:31,448 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-36) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@1eb1b7bd >2014-10-22 14:50:31,448 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-36) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 14:50:31,449 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-36) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 14:50:31,449 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-36) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 14:50:31,465 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-36) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 14:50:31,470 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-36) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 14:50:34,479 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 14:50:34,481 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:111) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-22 14:50:34,481 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:111) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-22 14:50:34,482 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-88) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:122) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:111) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-22 14:50:34,486 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-88) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-10-22 14:50:34,486 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-88) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 14:50:34,495 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-88) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:111) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-22 14:50:37,505 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 14:50:37,506 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:111) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-22 14:50:37,506 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:111) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-22 14:50:37,507 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-26) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:122) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:111) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-22 14:50:37,509 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-26) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-10-22 14:50:37,509 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-26) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 14:50:37,522 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-26) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:111) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-22 14:50:40,531 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 14:50:40,532 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:111) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-22 14:50:40,533 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:111) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-22 14:50:40,533 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-91) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:122) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:111) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-22 14:50:40,535 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-91) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-10-22 14:50:40,536 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-91) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 14:50:40,545 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-91) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:111) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-22 14:50:43,555 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 14:50:43,556 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:111) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-22 14:50:43,556 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:111) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-22 14:50:43,556 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-52) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:122) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:67) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:111) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-22 14:50:43,559 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-52) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-10-22 14:50:43,559 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-52) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 14:50:43,568 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-52) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:111) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$2.call(ReactorClient.java:97) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-10-22 14:50:46,578 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 14:50:46,587 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 14:53:49,029 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-66) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@d440115 >2014-10-22 14:53:49,029 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-66) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 14:53:49,029 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-66) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 14:53:49,029 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-66) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 14:53:49,045 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-66) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 14:53:49,058 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-66) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 14:53:52,325 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 14:53:52,351 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 14:56:54,603 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-50) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@6fc559e5 >2014-10-22 14:56:54,603 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-50) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 14:56:54,604 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-50) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 14:56:54,604 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-50) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 14:56:54,610 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-50) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 14:56:54,615 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-50) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 14:56:57,624 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 14:56:57,641 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 14:59:59,667 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-92) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@ac0afc9 >2014-10-22 14:59:59,667 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-92) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 14:59:59,667 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-92) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 14:59:59,668 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-92) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 14:59:59,705 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-92) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 14:59:59,709 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-92) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 15:00:02,852 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 15:00:02,878 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 15:03:05,235 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-4) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@3df60bef >2014-10-22 15:03:05,235 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-4) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 15:03:05,235 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-4) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 15:03:05,236 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-4) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 15:03:05,248 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 15:03:05,255 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-4) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 15:03:08,265 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 15:03:08,273 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 15:04:14,603 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-93) Attempting to update VMs/Templates Ovf. >2014-10-22 15:06:10,307 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-52) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@5f1f8e0c >2014-10-22 15:06:10,307 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-52) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 15:06:10,308 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-52) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 15:06:10,308 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-52) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 15:06:10,315 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-52) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 15:06:10,320 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-52) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 15:06:13,466 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 15:06:13,473 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 15:09:15,878 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-75) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@f885c34 >2014-10-22 15:09:15,878 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-75) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 15:09:15,879 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-75) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 15:09:15,879 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-75) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 15:09:15,891 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-75) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 15:09:15,897 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-75) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 15:09:18,907 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 15:09:18,913 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 15:10:42,935 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-8) [5c27910a] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 15:10:58,779 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-87) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 15:12:20,942 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-2) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@ce5f811 >2014-10-22 15:12:20,942 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-2) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 15:12:20,942 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-2) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 15:12:20,942 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-2) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 15:12:20,954 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-2) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 15:12:20,960 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-2) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 15:12:24,363 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 15:12:24,375 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 15:15:26,496 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-71) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@367f948b >2014-10-22 15:15:26,497 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-71) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 15:15:26,497 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-71) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 15:15:26,497 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-71) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 15:15:26,504 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-71) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 15:15:26,509 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-71) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 15:15:29,519 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 15:15:29,526 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 15:18:31,561 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-8) [5c27910a] Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@758dac32 >2014-10-22 15:18:31,561 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-8) [5c27910a] HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 15:18:31,561 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-8) [5c27910a] Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 15:18:31,561 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-8) [5c27910a] Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 15:18:31,577 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-8) [5c27910a] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 15:18:31,585 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-8) [5c27910a] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 15:18:34,979 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 15:18:35,016 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 15:21:37,123 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-2) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@38e04733 >2014-10-22 15:21:37,123 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-2) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 15:21:37,123 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-2) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 15:21:37,123 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-2) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 15:21:37,148 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-2) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 15:21:37,153 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-2) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 15:21:40,162 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 15:21:40,169 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 15:24:42,191 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-78) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@7b1c8bbc >2014-10-22 15:24:42,191 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-78) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 15:24:42,192 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-78) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 15:24:42,192 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-78) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 15:24:42,201 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-78) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 15:24:42,206 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-78) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 15:24:45,762 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 15:24:45,780 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 15:27:48,258 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-38) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@4f59bc38 >2014-10-22 15:27:48,258 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-38) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 15:27:48,258 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-38) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 15:27:48,259 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-38) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 15:27:48,265 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-38) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 15:27:48,270 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-38) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 15:27:51,280 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 15:27:51,287 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 15:30:53,318 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-63) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@2dca752a >2014-10-22 15:30:53,318 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-63) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 15:30:53,318 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-63) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 15:30:53,319 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-63) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 15:30:53,326 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-63) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 15:30:53,332 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-63) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 15:30:56,472 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 15:30:56,484 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 15:33:58,892 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-27) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@467ba590 >2014-10-22 15:33:58,892 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-27) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 15:33:58,892 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-27) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 15:33:58,892 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-27) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 15:33:58,901 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-27) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 15:33:58,905 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-27) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 15:34:01,914 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 15:34:01,933 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 15:37:03,966 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-7) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@6606252c >2014-10-22 15:37:03,966 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-7) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 15:37:03,966 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-7) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 15:37:03,967 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-7) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 15:37:03,974 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 15:37:03,979 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-7) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 15:37:07,383 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 15:37:07,389 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 15:40:09,543 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-60) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@32d8759b >2014-10-22 15:40:09,543 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-60) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 15:40:09,543 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-60) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 15:40:09,543 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-60) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 15:40:09,551 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-60) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 15:40:09,557 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-60) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 15:40:12,567 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 15:40:12,594 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 15:43:14,609 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-25) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@45594581 >2014-10-22 15:43:14,610 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-25) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 15:43:14,610 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-25) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 15:43:14,610 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-25) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 15:43:14,620 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-25) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 15:43:14,626 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-25) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 15:43:17,767 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 15:43:17,791 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 15:46:20,182 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-95) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@f7de6e3 >2014-10-22 15:46:20,182 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-95) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 15:46:20,184 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-95) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 15:46:20,184 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-95) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 15:46:20,192 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-95) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 15:46:20,198 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-95) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 15:46:23,208 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 15:46:23,219 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 15:49:25,251 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-28) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@ccd32f6 >2014-10-22 15:49:25,252 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-28) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 15:49:25,252 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-28) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 15:49:25,253 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-28) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 15:49:25,265 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-28) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 15:49:25,270 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-28) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 15:49:28,410 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 15:49:28,438 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 15:52:30,810 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-22) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@34d4d7ce >2014-10-22 15:52:30,811 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-22) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 15:52:30,811 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-22) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 15:52:30,811 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-22) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 15:52:30,818 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-22) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 15:52:30,824 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-22) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 15:52:33,833 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 15:52:33,841 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 15:55:35,872 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-95) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@16a1af40 >2014-10-22 15:55:35,872 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-95) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 15:55:35,872 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-95) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 15:55:35,873 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-95) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 15:55:35,880 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-95) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 15:55:35,885 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-95) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 15:55:39,283 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 15:55:39,295 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 15:58:41,440 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-34) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@618d67e2 >2014-10-22 15:58:41,440 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-34) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 15:58:41,440 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-34) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 15:58:41,440 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-34) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 15:58:41,455 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-34) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 15:58:41,460 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-34) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 15:58:44,476 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 15:58:44,502 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 16:01:46,514 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-22) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@83c4f47 >2014-10-22 16:01:46,514 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-22) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 16:01:46,514 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-22) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 16:01:46,514 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-22) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 16:01:46,522 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-22) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 16:01:46,527 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-22) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 16:01:49,677 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 16:01:49,687 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 16:04:14,618 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-25) Attempting to update VMs/Templates Ovf. >2014-10-22 16:04:52,075 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-40) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@327d859d >2014-10-22 16:04:52,075 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-40) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 16:04:52,075 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-40) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 16:04:52,076 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-40) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 16:04:52,083 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-40) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 16:04:52,089 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-40) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 16:04:55,101 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 16:04:55,115 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 16:07:57,142 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-24) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@d4496f4 >2014-10-22 16:07:57,142 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-24) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 16:07:57,142 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-24) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 16:07:57,143 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-24) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 16:07:57,150 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-24) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 16:07:57,154 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-24) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 16:08:00,295 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 16:08:00,302 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 16:10:49,861 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) [1883f47f] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 16:11:00,233 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-72) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 16:11:02,723 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-50) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@31b73b7a >2014-10-22 16:11:02,723 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-50) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 16:11:02,723 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-50) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 16:11:02,723 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-50) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 16:11:02,730 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-50) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 16:11:02,734 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-50) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 16:11:05,745 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 16:11:05,754 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 16:14:07,792 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-70) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@5d291c9 >2014-10-22 16:14:07,792 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-70) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 16:14:07,793 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-70) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 16:14:07,793 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-70) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 16:14:07,801 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-70) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 16:14:07,805 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-70) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 16:14:10,945 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 16:14:10,953 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 16:17:13,349 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-95) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@60bc4ced >2014-10-22 16:17:13,349 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-95) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 16:17:13,349 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-95) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 16:17:13,350 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-95) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 16:17:13,357 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-95) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 16:17:13,362 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-95) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 16:17:16,380 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 16:17:16,389 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 16:20:18,418 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-53) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@690a0c96 >2014-10-22 16:20:18,418 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-53) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 16:20:18,418 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-53) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 16:20:18,418 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-53) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 16:20:18,426 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-53) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 16:20:18,431 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-53) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 16:20:21,570 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma07.qa.lab.tlv.redhat.com/10.35.117.34 >2014-10-22 16:20:21,588 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1290) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:513) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1180) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1152) [jsse.jar:1.7.0_65] > at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:109) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:134) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] >Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) [jsse.jar:1.7.0_65] > at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1694) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:278) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:808) [jsse.jar:1.7.0_65] > at sun.security.ssl.Handshaker$1.run(Handshaker.java:806) [jsse.jar:1.7.0_65] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_65] > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1227) [jsse.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient$1.call(SSLClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] >Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) [rt.jar:1.7.0_65] > at sun.security.validator.Validator.validate(Validator.java:260) [rt.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) [jsse.jar:1.7.0_65] > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) [jsse.jar:1.7.0_65] > at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1328) [jsse.jar:1.7.0_65] > ... 11 more >Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196) [rt.jar:1.7.0_65] > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268) [rt.jar:1.7.0_65] > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380) [rt.jar:1.7.0_65] > ... 17 more > >2014-10-22 16:21:19,129 INFO [org.ovirt.engine.core.bll.aaa.LoginBaseCommand] (ajp-/127.0.0.1:8702-5) Cant login user "admin" with authentication profile "internal" because the authentication failed. >2014-10-22 16:21:19,133 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin cannot login, please verify the username and password. >2014-10-22 16:21:19,136 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin failed to log in. >2014-10-22 16:21:19,136 WARN [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (ajp-/127.0.0.1:8702-5) CanDoAction of action LoginAdminUser failed. Reasons:USER_FAILED_TO_AUTHENTICATE_WRONG_USERNAME_OR_PASSWORD >2014-10-22 16:21:21,398 INFO [org.ovirt.engine.core.bll.aaa.LoginBaseCommand] (ajp-/127.0.0.1:8702-2) Cant login user "admin" with authentication profile "internal" because the authentication failed. >2014-10-22 16:21:21,406 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin cannot login, please verify the username and password. >2014-10-22 16:21:21,408 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin failed to log in. >2014-10-22 16:21:21,408 WARN [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (ajp-/127.0.0.1:8702-2) CanDoAction of action LoginAdminUser failed. Reasons:USER_FAILED_TO_AUTHENTICATE_WRONG_USERNAME_OR_PASSWORD >2014-10-22 16:21:25,170 INFO [org.ovirt.engine.core.bll.aaa.LoginBaseCommand] (ajp-/127.0.0.1:8702-1) Cant login user "admin" with authentication profile "internal" because the authentication failed. >2014-10-22 16:21:25,174 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin cannot login, please verify the username and password. >2014-10-22 16:21:25,176 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin failed to log in. >2014-10-22 16:21:25,177 WARN [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (ajp-/127.0.0.1:8702-1) CanDoAction of action LoginAdminUser failed. Reasons:USER_FAILED_TO_AUTHENTICATE_WRONG_USERNAME_OR_PASSWORD >2014-10-22 16:21:29,998 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-12) Invalid locale found in configuration: >2014-10-22 16:21:29,999 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-12) Invalid locale found in configuration: >2014-10-22 16:21:33,735 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (ajp-/127.0.0.1:8702-11) Running command: LoginAdminUserCommand internal: false. >2014-10-22 16:21:33,746 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-10-22 16:21:33,871 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-3) Running command: LoginUserCommand internal: false. >2014-10-22 16:21:53,611 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-7-thread-32) [4af1817] Start running CanDoAction for command number 2/2 (Command type: RemoveVmFromPool) >2014-10-22 16:21:53,611 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-7-thread-17) [7f0e6fcc] Start running CanDoAction for command number 1/2 (Command type: RemoveVmFromPool) >2014-10-22 16:21:53,621 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-7-thread-17) [7f0e6fcc] Finish handling CanDoAction for command number 1/2 (Command type: RemoveVmFromPool) >2014-10-22 16:21:53,621 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-7-thread-32) [4af1817] Finish handling CanDoAction for command number 2/2 (Command type: RemoveVmFromPool) >2014-10-22 16:21:53,622 INFO [org.ovirt.engine.core.bll.RemoveVmFromPoolCommand] (org.ovirt.thread.pool-7-thread-1) [7f0e6fcc] Running command: RemoveVmFromPoolCommand internal: false. Entities affected : ID: 8f1febe6-db76-4ff6-ab30-7f42cb8cf781 Type: VmPoolAction group EDIT_VM_POOL_CONFIGURATION with role type USER >2014-10-22 16:21:53,631 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-1) [7f0e6fcc] Correlation ID: 7f0e6fcc, Call Stack: null, Custom Event ID: -1, Message: VM ppc_pool-1 was removed from VM Pool ppc_pool by admin. >2014-10-22 16:21:53,632 INFO [org.ovirt.engine.core.bll.RemoveVmFromPoolCommand] (org.ovirt.thread.pool-7-thread-1) [4af1817] Running command: RemoveVmFromPoolCommand internal: false. Entities affected : ID: 8f1febe6-db76-4ff6-ab30-7f42cb8cf781 Type: VmPoolAction group EDIT_VM_POOL_CONFIGURATION with role type USER >2014-10-22 16:21:53,641 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-1) [4af1817] Correlation ID: 4af1817, Call Stack: null, Custom Event ID: -1, Message: VM ppc_pool-2 was removed from VM Pool ppc_pool by admin. >2014-10-22 16:21:53,652 INFO [org.ovirt.engine.core.bll.RemoveVmPoolCommand] (org.ovirt.thread.pool-7-thread-1) [78094c0a] Running command: RemoveVmPoolCommand internal: true. Entities affected : ID: 8f1febe6-db76-4ff6-ab30-7f42cb8cf781 Type: VmPoolAction group DELETE_VM_POOL with role type USER >2014-10-22 16:21:53,670 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-1) [78094c0a] Correlation ID: 78094c0a, Call Stack: null, Custom Event ID: -1, Message: VM Pool ppc_pool was removed by admin. >2014-10-22 16:22:05,592 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-7-thread-12) [83c59f6] Start running CanDoAction for command number 2/3 (Command type: RemoveVm) >2014-10-22 16:22:05,592 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-7-thread-13) [73038b53] Start running CanDoAction for command number 3/3 (Command type: RemoveVm) >2014-10-22 16:22:05,592 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-7-thread-43) [5cab6447] Start running CanDoAction for command number 1/3 (Command type: RemoveVm) >2014-10-22 16:22:05,594 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-13) [73038b53] Lock Acquired to object EngineLock [exclusiveLocks= key: 3c71926d-02cd-46fa-8975-f0986ca49674 value: VM >, sharedLocks= ] >2014-10-22 16:22:05,594 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-12) [83c59f6] Lock Acquired to object EngineLock [exclusiveLocks= key: e1c392c2-1ccc-433a-9f30-ea302d6ece77 value: VM >, sharedLocks= ] >2014-10-22 16:22:05,596 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-43) [5cab6447] Lock Acquired to object EngineLock [exclusiveLocks= key: 42998ef0-6fc3-4e28-bf56-b388913b547d value: VM >, sharedLocks= ] >2014-10-22 16:22:05,689 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-7-thread-12) [83c59f6] Finish handling CanDoAction for command number 2/3 (Command type: RemoveVm) >2014-10-22 16:22:05,750 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-7-thread-13) [73038b53] Finish handling CanDoAction for command number 3/3 (Command type: RemoveVm) >2014-10-22 16:22:05,772 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-7-thread-43) [5cab6447] Finish handling CanDoAction for command number 1/3 (Command type: RemoveVm) >2014-10-22 16:22:05,801 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-5) [5cab6447] Running command: RemoveVmCommand internal: false. Entities affected : ID: 42998ef0-6fc3-4e28-bf56-b388913b547d Type: VMAction group DELETE_VM with role type USER >2014-10-22 16:22:05,803 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-5) [5cab6447] START, SetVmStatusVDSCommand( vmId = 42998ef0-6fc3-4e28-bf56-b388913b547d, status = ImageLocked, exit status = Normal), log id: a4089c8 >2014-10-22 16:22:05,807 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-5) [5cab6447] FINISH, SetVmStatusVDSCommand, log id: a4089c8 >2014-10-22 16:22:05,809 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-5) [5cab6447] Lock freed to object EngineLock [exclusiveLocks= key: 42998ef0-6fc3-4e28-bf56-b388913b547d value: VM >, sharedLocks= ] >2014-10-22 16:22:05,833 INFO [org.ovirt.engine.core.bll.RemoveAllVmImagesCommand] (org.ovirt.thread.pool-7-thread-5) [661e11a] Running command: RemoveAllVmImagesCommand internal: true. Entities affected : ID: 42998ef0-6fc3-4e28-bf56-b388913b547d Type: VM >2014-10-22 16:22:05,866 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-7-thread-5) [1f6af7a2] Running command: RemoveImageCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage >2014-10-22 16:22:05,946 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-7-thread-5) [1f6af7a2] START, DeleteImageGroupVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = 075b5199-62ff-44cd-8b8e-e1eccedfcb6c, postZeros = false, forceDelete = false), log id: eb67501 >2014-10-22 16:22:06,215 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-7-thread-5) [1f6af7a2] FINISH, DeleteImageGroupVDSCommand, log id: eb67501 >2014-10-22 16:22:06,220 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-5) [1f6af7a2] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 5e129c08-2fbb-4875-8af5-6f718fbc97cb >2014-10-22 16:22:06,220 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-7-thread-5) [1f6af7a2] CommandMultiAsyncTasks::AttachTask: Attaching task adcd9f95-023f-46a7-ba50-29e9d09168f4 to command 5e129c08-2fbb-4875-8af5-6f718fbc97cb. >2014-10-22 16:22:06,231 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-7-thread-5) [1f6af7a2] Adding task adcd9f95-023f-46a7-ba50-29e9d09168f4 (Parent Command RemoveVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-10-22 16:22:06,360 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-5) [1f6af7a2] Correlation ID: 5cab6447, Job ID: 5de11d2e-e31d-4693-be19-ecfcd7cc0a23, Call Stack: null, Custom Event ID: -1, Message: VM ppc_pool-1 was successfully removed. >2014-10-22 16:22:06,360 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-5) [1f6af7a2] BaseAsyncTask::startPollingTask: Starting to poll task adcd9f95-023f-46a7-ba50-29e9d09168f4. >2014-10-22 16:22:06,382 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-5) [83c59f6] Running command: RemoveVmCommand internal: false. Entities affected : ID: e1c392c2-1ccc-433a-9f30-ea302d6ece77 Type: VMAction group DELETE_VM with role type USER >2014-10-22 16:22:06,383 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-5) [83c59f6] START, SetVmStatusVDSCommand( vmId = e1c392c2-1ccc-433a-9f30-ea302d6ece77, status = ImageLocked, exit status = Normal), log id: 56cd16eb >2014-10-22 16:22:06,387 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-5) [83c59f6] FINISH, SetVmStatusVDSCommand, log id: 56cd16eb >2014-10-22 16:22:06,389 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-5) [83c59f6] Lock freed to object EngineLock [exclusiveLocks= key: e1c392c2-1ccc-433a-9f30-ea302d6ece77 value: VM >, sharedLocks= ] >2014-10-22 16:22:06,419 INFO [org.ovirt.engine.core.bll.RemoveAllVmImagesCommand] (org.ovirt.thread.pool-7-thread-5) [18e9ba75] Running command: RemoveAllVmImagesCommand internal: true. Entities affected : ID: e1c392c2-1ccc-433a-9f30-ea302d6ece77 Type: VM >2014-10-22 16:22:06,434 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-7-thread-5) [6180cb8] Running command: RemoveImageCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage >2014-10-22 16:22:06,451 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-7-thread-5) [6180cb8] START, DeleteImageGroupVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = ff51fcdc-f313-49ef-a677-dfa8cbc00820, postZeros = false, forceDelete = false), log id: 29fceb57 >2014-10-22 16:22:06,736 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-7-thread-5) [6180cb8] FINISH, DeleteImageGroupVDSCommand, log id: 29fceb57 >2014-10-22 16:22:06,741 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-5) [6180cb8] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 00fbe193-213e-482d-a2cd-9003232c024d >2014-10-22 16:22:06,741 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-7-thread-5) [6180cb8] CommandMultiAsyncTasks::AttachTask: Attaching task 09e23996-238d-4339-a3ac-d66cd3012c4e to command 00fbe193-213e-482d-a2cd-9003232c024d. >2014-10-22 16:22:06,751 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-7-thread-5) [6180cb8] Adding task 09e23996-238d-4339-a3ac-d66cd3012c4e (Parent Command RemoveVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-10-22 16:22:06,900 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-5) [6180cb8] Correlation ID: 83c59f6, Job ID: b9e6dbab-2a02-4146-a9e2-23d68d07eca1, Call Stack: null, Custom Event ID: -1, Message: VM ppc_pool-2 was successfully removed. >2014-10-22 16:22:06,900 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-5) [6180cb8] BaseAsyncTask::startPollingTask: Starting to poll task 09e23996-238d-4339-a3ac-d66cd3012c4e. >2014-10-22 16:22:06,918 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-5) [73038b53] Running command: RemoveVmCommand internal: false. Entities affected : ID: 3c71926d-02cd-46fa-8975-f0986ca49674 Type: VMAction group DELETE_VM with role type USER >2014-10-22 16:22:06,919 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-5) [73038b53] START, SetVmStatusVDSCommand( vmId = 3c71926d-02cd-46fa-8975-f0986ca49674, status = ImageLocked, exit status = Normal), log id: 40819e48 >2014-10-22 16:22:06,922 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-5) [73038b53] FINISH, SetVmStatusVDSCommand, log id: 40819e48 >2014-10-22 16:22:06,924 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-5) [73038b53] Lock freed to object EngineLock [exclusiveLocks= key: 3c71926d-02cd-46fa-8975-f0986ca49674 value: VM >, sharedLocks= ] >2014-10-22 16:22:06,963 INFO [org.ovirt.engine.core.bll.RemoveAllVmImagesCommand] (org.ovirt.thread.pool-7-thread-5) [20ac48c] Running command: RemoveAllVmImagesCommand internal: true. Entities affected : ID: 3c71926d-02cd-46fa-8975-f0986ca49674 Type: VM >2014-10-22 16:22:06,970 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-7-thread-5) [734fb4ce] Running command: RemoveImageCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage >2014-10-22 16:22:07,110 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-7-thread-5) [734fb4ce] START, DeleteImageGroupVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = 4323cfe2-a20f-48d0-8d3c-d02e52d43625, postZeros = false, forceDelete = false), log id: 628ccf5a >2014-10-22 16:22:07,270 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-7-thread-5) [734fb4ce] FINISH, DeleteImageGroupVDSCommand, log id: 628ccf5a >2014-10-22 16:22:07,275 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-5) [734fb4ce] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command c7963f96-2672-425b-a30a-aa6a47b25c75 >2014-10-22 16:22:07,276 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-7-thread-5) [734fb4ce] CommandMultiAsyncTasks::AttachTask: Attaching task 28674ec3-bd29-4e97-aa0c-e47f20258523 to command c7963f96-2672-425b-a30a-aa6a47b25c75. >2014-10-22 16:22:07,289 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-7-thread-5) [734fb4ce] Adding task 28674ec3-bd29-4e97-aa0c-e47f20258523 (Parent Command RemoveVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-10-22 16:22:07,347 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-5) [734fb4ce] Correlation ID: 73038b53, Job ID: c87df83d-6380-43ec-b010-ee43d6dce79a, Call Stack: null, Custom Event ID: -1, Message: VM ppc_vm was successfully removed. >2014-10-22 16:22:07,347 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-5) [734fb4ce] BaseAsyncTask::startPollingTask: Starting to poll task 28674ec3-bd29-4e97-aa0c-e47f20258523. >2014-10-22 16:22:16,047 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-88) Polling and updating Async Tasks: 3 tasks, 3 tasks to poll now >2014-10-22 16:22:16,059 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-88) SPMAsyncTask::PollTask: Polling task adcd9f95-023f-46a7-ba50-29e9d09168f4 (Parent Command RemoveVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-10-22 16:22:16,063 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-88) BaseAsyncTask::onTaskEndSuccess: Task adcd9f95-023f-46a7-ba50-29e9d09168f4 (Parent Command RemoveVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-10-22 16:22:16,064 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-88) CommandAsyncTask::endActionIfNecessary: All tasks of command 5e129c08-2fbb-4875-8af5-6f718fbc97cb has ended -> executing endAction >2014-10-22 16:22:16,064 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-88) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: 5e129c08-2fbb-4875-8af5-6f718fbc97cb): calling endAction . >2014-10-22 16:22:16,065 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-88) SPMAsyncTask::PollTask: Polling task 28674ec3-bd29-4e97-aa0c-e47f20258523 (Parent Command RemoveVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-10-22 16:22:16,065 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-4) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction RemoveVm, executionIndex: 0 >2014-10-22 16:22:16,076 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-88) BaseAsyncTask::onTaskEndSuccess: Task 28674ec3-bd29-4e97-aa0c-e47f20258523 (Parent Command RemoveVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-10-22 16:22:16,077 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-88) CommandAsyncTask::endActionIfNecessary: All tasks of command c7963f96-2672-425b-a30a-aa6a47b25c75 has ended -> executing endAction >2014-10-22 16:22:16,077 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-88) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: c7963f96-2672-425b-a30a-aa6a47b25c75): calling endAction . >2014-10-22 16:22:16,078 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-88) SPMAsyncTask::PollTask: Polling task 09e23996-238d-4339-a3ac-d66cd3012c4e (Parent Command RemoveVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-10-22 16:22:16,079 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-39) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction RemoveVm, executionIndex: 0 >2014-10-22 16:22:16,093 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-88) BaseAsyncTask::onTaskEndSuccess: Task 09e23996-238d-4339-a3ac-d66cd3012c4e (Parent Command RemoveVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-10-22 16:22:16,093 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-4) Ending command successfully: org.ovirt.engine.core.bll.RemoveVmCommand >2014-10-22 16:22:16,094 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-88) CommandAsyncTask::endActionIfNecessary: All tasks of command 00fbe193-213e-482d-a2cd-9003232c024d has ended -> executing endAction >2014-10-22 16:22:16,094 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-88) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: 00fbe193-213e-482d-a2cd-9003232c024d): calling endAction . >2014-10-22 16:22:16,095 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-4) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RemoveVm completed, handling the result. >2014-10-22 16:22:16,096 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-4) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RemoveVm succeeded, clearing tasks. >2014-10-22 16:22:16,100 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-24) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction RemoveVm, executionIndex: 0 >2014-10-22 16:22:16,111 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-39) Ending command successfully: org.ovirt.engine.core.bll.RemoveVmCommand >2014-10-22 16:22:16,112 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-4) SPMAsyncTask::ClearAsyncTask: Attempting to clear task adcd9f95-023f-46a7-ba50-29e9d09168f4 >2014-10-22 16:22:16,113 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-4) START, SPMClearTaskVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, taskId = adcd9f95-023f-46a7-ba50-29e9d09168f4), log id: 513469be >2014-10-22 16:22:16,113 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-39) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RemoveVm completed, handling the result. >2014-10-22 16:22:16,113 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-39) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RemoveVm succeeded, clearing tasks. >2014-10-22 16:22:16,118 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-39) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 28674ec3-bd29-4e97-aa0c-e47f20258523 >2014-10-22 16:22:16,119 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-39) START, SPMClearTaskVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, taskId = 28674ec3-bd29-4e97-aa0c-e47f20258523), log id: 1a098e57 >2014-10-22 16:22:16,119 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-4) START, HSMClearTaskVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, taskId=adcd9f95-023f-46a7-ba50-29e9d09168f4), log id: 480a6298 >2014-10-22 16:22:16,121 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-24) Ending command successfully: org.ovirt.engine.core.bll.RemoveVmCommand >2014-10-22 16:22:16,122 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-24) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RemoveVm completed, handling the result. >2014-10-22 16:22:16,122 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-24) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RemoveVm succeeded, clearing tasks. >2014-10-22 16:22:16,125 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-24) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 09e23996-238d-4339-a3ac-d66cd3012c4e >2014-10-22 16:22:16,126 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-24) START, SPMClearTaskVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, taskId = 09e23996-238d-4339-a3ac-d66cd3012c4e), log id: 14b0ea2f >2014-10-22 16:22:16,156 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-4) FINISH, HSMClearTaskVDSCommand, log id: 480a6298 >2014-10-22 16:22:16,156 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-4) FINISH, SPMClearTaskVDSCommand, log id: 513469be >2014-10-22 16:22:16,162 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-24) START, HSMClearTaskVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, taskId=09e23996-238d-4339-a3ac-d66cd3012c4e), log id: 7f71825 >2014-10-22 16:22:16,163 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-4) BaseAsyncTask::removeTaskFromDB: Removed task adcd9f95-023f-46a7-ba50-29e9d09168f4 from DataBase >2014-10-22 16:22:16,163 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-4) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 5e129c08-2fbb-4875-8af5-6f718fbc97cb >2014-10-22 16:22:16,194 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-24) FINISH, HSMClearTaskVDSCommand, log id: 7f71825 >2014-10-22 16:22:16,194 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-24) FINISH, SPMClearTaskVDSCommand, log id: 14b0ea2f >2014-10-22 16:22:16,195 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-39) START, HSMClearTaskVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, taskId=28674ec3-bd29-4e97-aa0c-e47f20258523), log id: 2384869f >2014-10-22 16:22:16,199 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-24) BaseAsyncTask::removeTaskFromDB: Removed task 09e23996-238d-4339-a3ac-d66cd3012c4e from DataBase >2014-10-22 16:22:16,199 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-24) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 00fbe193-213e-482d-a2cd-9003232c024d >2014-10-22 16:22:16,223 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-39) FINISH, HSMClearTaskVDSCommand, log id: 2384869f >2014-10-22 16:22:16,223 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-39) FINISH, SPMClearTaskVDSCommand, log id: 1a098e57 >2014-10-22 16:22:16,228 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-39) BaseAsyncTask::removeTaskFromDB: Removed task 28674ec3-bd29-4e97-aa0c-e47f20258523 from DataBase >2014-10-22 16:22:16,229 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-39) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity c7963f96-2672-425b-a30a-aa6a47b25c75 >2014-10-22 16:22:17,654 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-7-thread-9) [52b65433] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: d5b2492d-8110-4614-9ce4-4a7a118c7942 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-10-22 16:22:17,736 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (org.ovirt.thread.pool-7-thread-9) [52b65433] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: d5b2492d-8110-4614-9ce4-4a7a118c7942 Type: VDS >2014-10-22 16:22:17,743 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-9) [52b65433] START, SetVdsStatusVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, status=Maintenance, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 592efe15 >2014-10-22 16:23:24,031 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-23) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@657b693 >2014-10-22 16:23:24,032 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-23) HostName = alma07.qa.lab.tlv.redhat.com >2014-10-22 16:23:24,032 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-23) Command GetCapabilitiesVDSCommand(HostName = alma07.qa.lab.tlv.redhat.com, HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942, vds=Host[alma07.qa.lab.tlv.redhat.com,d5b2492d-8110-4614-9ce4-4a7a118c7942]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-10-22 16:23:24,032 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-23) Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 16:23:24,039 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-23) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-10-22 16:23:24,044 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-23) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:571) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:657) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:504) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-10-22 16:23:24,051 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-7-thread-9) [52b65433] FINISH, SetVdsStatusVDSCommand, log id: 592efe15 >2014-10-22 16:23:24,056 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-9) [52b65433] Correlation ID: 52b65433, Job ID: 63f5c4a7-d8bc-4aae-82d1-ec23c40f0c23, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com was switched to Maintenance mode by admin. >2014-10-22 16:23:34,501 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (ajp-/127.0.0.1:8702-10) [2c245f72] Lock Acquired to object EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS >, sharedLocks= ] >2014-10-22 16:23:34,515 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (org.ovirt.thread.pool-7-thread-33) [2c245f72] Running command: RemoveVdsCommand internal: false. Entities affected : ID: d5b2492d-8110-4614-9ce4-4a7a118c7942 Type: VDSAction group DELETE_HOST with role type ADMIN >2014-10-22 16:23:34,559 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (org.ovirt.thread.pool-7-thread-33) [2c245f72] START, RemoveVdsVDSCommand( HostId = d5b2492d-8110-4614-9ce4-4a7a118c7942), log id: 3b6aba7d >2014-10-22 16:23:34,559 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (org.ovirt.thread.pool-7-thread-33) [2c245f72] vdsManager::disposing >2014-10-22 16:23:34,560 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (org.ovirt.thread.pool-7-thread-33) [2c245f72] FINISH, RemoveVdsVDSCommand, log id: 3b6aba7d >2014-10-22 16:23:34,566 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-33) [2c245f72] Correlation ID: 2c245f72, Call Stack: null, Custom Event ID: -1, Message: Host alma07.qa.lab.tlv.redhat.com was removed by admin. >2014-10-22 16:23:34,566 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (org.ovirt.thread.pool-7-thread-33) [2c245f72] Lock freed to object EngineLock [exclusiveLocks= key: d5b2492d-8110-4614-9ce4-4a7a118c7942 value: VDS >, sharedLocks= ] >2014-10-22 16:23:40,285 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-54) Setting new tasks map. The map contains now 0 tasks >2014-10-22 16:23:40,286 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-54) Cleared all tasks of pool f8f5eaee-8fd0-4b45-87db-62d61b03a916. >2014-10-22 16:29:29,520 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-38) Timer update runtimeinfo failed. Exception:: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000451: The connection manager is shutdown: java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:141) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:40) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:35) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:220) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000451: The connection manager is shutdown: java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:151) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more >Caused by: javax.resource.ResourceException: IJ000451: The connection manager is shutdown: java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:345) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:488) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:143) > ... 21 more > >2014-10-22 16:29:29,561 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-26) Timer update runtimeinfo failed. Exception:: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000451: The connection manager is shutdown: java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:141) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:40) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:35) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:220) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000451: The connection manager is shutdown: java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:151) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more >Caused by: javax.resource.ResourceException: IJ000451: The connection manager is shutdown: java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:345) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:488) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:143) > ... 21 more > >2014-10-22 16:32:01,355 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Loaded file "/usr/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf". >2014-10-22 16:32:01,407 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) The file "/etc/ovirt-engine/engine.conf" doesn't exist or isn't readable. Will return an empty set of properties. >2014-10-22 16:32:01,407 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Loaded file "/etc/ovirt-engine/engine.conf.d/10-rhevm-doc.conf". >2014-10-22 16:32:01,408 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-database.conf". >2014-10-22 16:32:01,408 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf". >2014-10-22 16:32:01,409 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf". >2014-10-22 16:32:01,409 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf". >2014-10-22 16:32:01,409 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Loaded file "/etc/ovirt-engine/engine.conf.d/20-branding-rhev.conf". >2014-10-22 16:32:01,410 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Loaded file "/etc/ovirt-engine/engine.conf.d/50-redhat-support-plugin.conf". >2014-10-22 16:32:01,411 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_AJP_ENABLED" is "true". >2014-10-22 16:32:01,411 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_AJP_PORT" is "8702". >2014-10-22 16:32:01,411 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_APPS" is "engine.ear restapi.war legacy_restapi.war "/usr/share/ovirt-engine/branding/rhev-2.brand/applications/rhev.ear" redhat-support-plugin-rhev.ear". >2014-10-22 16:32:01,411 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_CACHE" is "/var/cache/ovirt-engine". >2014-10-22 16:32:01,412 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_CHECK_INTERVAL" is "1000". >2014-10-22 16:32:01,412 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_CONNECTION_TIMEOUT" is "300000". >2014-10-22 16:32:01,412 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_DATABASE" is "engine". >2014-10-22 16:32:01,412 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_DRIVER" is "org.postgresql.Driver". >2014-10-22 16:32:01,412 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_HOST" is "localhost". >2014-10-22 16:32:01,413 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_MAX_CONNECTIONS" is "100". >2014-10-22 16:32:01,413 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_MIN_CONNECTIONS" is "1". >2014-10-22 16:32:01,413 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_PASSWORD" is "***". >2014-10-22 16:32:01,413 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_PORT" is "5432". >2014-10-22 16:32:01,413 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_SECURED" is "False". >2014-10-22 16:32:01,414 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_SECURED_VALIDATION" is "False". >2014-10-22 16:32:01,414 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_URL" is "jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory". >2014-10-22 16:32:01,414 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_USER" is "engine". >2014-10-22 16:32:01,414 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DEBUG_ADDRESS" is "". >2014-10-22 16:32:01,415 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DOC" is "/usr/share/doc/ovirt-engine". >2014-10-22 16:32:01,415 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_ETC" is "/etc/ovirt-engine". >2014-10-22 16:32:01,415 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_EXTENSION_PATH" is "/usr/share/ovirt-engine/extensions.d:/etc/ovirt-engine/extensions.d". >2014-10-22 16:32:01,415 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_FQDN" is "dhcp163-98.scl.lab.tlv.redhat.com". >2014-10-22 16:32:01,415 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_GROUP" is "ovirt". >2014-10-22 16:32:01,416 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_HEAP_MAX" is "1g". >2014-10-22 16:32:01,416 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_HEAP_MIN" is "1g". >2014-10-22 16:32:01,416 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_HTTPS_ENABLED" is "false". >2014-10-22 16:32:01,416 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_HTTPS_PORT" is "None". >2014-10-22 16:32:01,465 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_HTTPS_PROTOCOLS" is "SSLv3,TLSv1,TLSv1.1,TLSv1.2". >2014-10-22 16:32:01,465 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_HTTP_ENABLED" is "false". >2014-10-22 16:32:01,465 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_HTTP_PORT" is "None". >2014-10-22 16:32:01,466 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_JAVA_MODULEPATH" is "/usr/share/ovirt-engine/modules". >2014-10-22 16:32:01,466 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_JVM_ARGS" is " -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/var/log/ovirt-engine/dump"". >2014-10-22 16:32:01,466 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_LOG" is "/var/log/ovirt-engine". >2014-10-22 16:32:01,466 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_LOG_TO_CONSOLE" is "false". >2014-10-22 16:32:01,467 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_MANUAL" is "/usr/share/doc/rhevm-doc/manual". >2014-10-22 16:32:01,467 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PERM_MAX" is "256m". >2014-10-22 16:32:01,467 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PERM_MIN" is "256m". >2014-10-22 16:32:01,468 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PKI" is "/etc/pki/ovirt-engine". >2014-10-22 16:32:01,468 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PKI_CA" is "/etc/pki/ovirt-engine/ca.pem". >2014-10-22 16:32:01,468 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PKI_ENGINE_CERT" is "/etc/pki/ovirt-engine/certs/engine.cer". >2014-10-22 16:32:01,469 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PKI_ENGINE_STORE" is "/etc/pki/ovirt-engine/keys/engine.p12". >2014-10-22 16:32:01,469 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PKI_ENGINE_STORE_ALIAS" is "1". >2014-10-22 16:32:01,469 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PKI_ENGINE_STORE_PASSWORD" is "***". >2014-10-22 16:32:01,469 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PKI_TRUST_STORE" is "/etc/pki/ovirt-engine/.truststore". >2014-10-22 16:32:01,469 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PKI_TRUST_STORE_PASSWORD" is "***". >2014-10-22 16:32:01,470 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PROPERTIES" is " jsse.enableSNIExtension=false "java.security.krb5.conf=/etc/ovirt-engine/krb5.conf"". >2014-10-22 16:32:01,470 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PROXY_ENABLED" is "true". >2014-10-22 16:32:01,470 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PROXY_HTTPS_PORT" is "443". >2014-10-22 16:32:01,470 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PROXY_HTTP_PORT" is "80". >2014-10-22 16:32:01,470 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_REPORTS_BASE_URL" is "". >2014-10-22 16:32:01,471 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_REPORTS_DASHBOARD_URL" is "/flow.html?viewAsDashboardFrame=true". >2014-10-22 16:32:01,471 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_REPORTS_NOT_INSTALLED_URL" is "/ovirt-engine/ReportsNotInstalled.html". >2014-10-22 16:32:01,471 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_REPORTS_PROXY_URL" is "/ovirt/reports-interface". >2014-10-22 16:32:01,471 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_REPORTS_READ_TIMEOUT" is "". >2014-10-22 16:32:01,471 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_REPORTS_RIGHTCLICK_URL" is "". >2014-10-22 16:32:01,472 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_REPORTS_VERIFY_CHAIN" is "true". >2014-10-22 16:32:01,472 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_REPORTS_VERIFY_HOST" is "true". >2014-10-22 16:32:01,472 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_STOP_INTERVAL" is "1". >2014-10-22 16:32:01,472 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_STOP_TIME" is "10". >2014-10-22 16:32:01,472 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_TMP" is "/var/tmp/ovirt-engine". >2014-10-22 16:32:01,473 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_UP_MARK" is "/var/lib/ovirt-engine/engine.up". >2014-10-22 16:32:01,473 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_URI" is "/ovirt-engine". >2014-10-22 16:32:01,473 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_USER" is "ovirt". >2014-10-22 16:32:01,473 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_USR" is "/usr/share/ovirt-engine". >2014-10-22 16:32:01,473 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_VAR" is "/var/lib/ovirt-engine". >2014-10-22 16:32:01,474 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_VERBOSE_GC" is "false". >2014-10-22 16:32:01,474 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "JBOSS_HOME" is "/usr/share/jbossas". >2014-10-22 16:32:01,474 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "JBOSS_RUNTIME" is "/var/lib/ovirt-engine/jboss_runtime". >2014-10-22 16:32:01,474 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "SENSITIVE_KEYS" is ",ENGINE_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD". >2014-10-22 16:32:15,904 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 46) Start initializing Backend >2014-10-22 16:32:17,249 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (ServerService Thread Pool -- 46) Loaded file /etc/ovirt-engine/osinfo.conf.d/00-defaults.properties >2014-10-22 16:32:17,600 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 46) Running ovirt-engine 3.5.0-0.17.beta.el6ev >2014-10-22 16:32:17,600 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ServerService Thread Pool -- 46) Start initializing dictionaries >2014-10-22 16:32:17,604 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ServerService Thread Pool -- 46) Finished initializing dictionaries >2014-10-22 16:32:17,605 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 46) Start initializing AuditLogCleanupManager >2014-10-22 16:32:17,605 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 46) Setting audit cleanup manager to run at: 35 35 3 * * ? >2014-10-22 16:32:17,698 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 46) Finished initializing AuditLogCleanupManager >2014-10-22 16:32:17,699 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 46) Start initializing CommandEntityCleanupManager >2014-10-22 16:32:17,699 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 46) Setting command entity cleanup manager to run at: 35 35 3 * * ? >2014-10-22 16:32:17,701 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 46) Finished initializing CommandEntityCleanupManager >2014-10-22 16:32:17,725 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 46) Start initializing TagsDirector >2014-10-22 16:32:17,753 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 46) Tag root added to tree >2014-10-22 16:32:17,806 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 46) Finished initializing TagsDirector >2014-10-22 16:32:17,807 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (ServerService Thread Pool -- 46) Start initializing IsoDomainListSyncronizer >2014-10-22 16:32:17,816 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (ServerService Thread Pool -- 46) Finished initializing IsoDomainListSyncronizer >2014-10-22 16:32:17,926 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 46) Completed initializing handlers >2014-10-22 16:32:18,057 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 46) Start initializing ErrorTranslatorImpl >2014-10-22 16:32:18,116 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 46) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table. >2014-10-22 16:32:18,118 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 46) Finished initializing ErrorTranslatorImpl >2014-10-22 16:32:18,118 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 46) Start initializing ErrorTranslatorImpl >2014-10-22 16:32:18,118 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 46) Finished initializing ErrorTranslatorImpl >2014-10-22 16:32:18,119 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 46) Mark incomplete jobs as UNKNOWN >2014-10-22 16:32:18,277 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ServerService Thread Pool -- 46) Start initializing JobRepositoryCleanupManager >2014-10-22 16:32:18,278 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ServerService Thread Pool -- 46) Finished initializing JobRepositoryCleanupManager >2014-10-22 16:32:18,279 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ServerService Thread Pool -- 46) Start initializing AutoRecoveryManager >2014-10-22 16:32:18,291 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ServerService Thread Pool -- 46) Finished initializing AutoRecoveryManager >2014-10-22 16:32:18,293 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ServerService Thread Pool -- 46) Start initializing ExecutionMessageDirector >2014-10-22 16:32:18,305 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ServerService Thread Pool -- 46) Finished initializing ExecutionMessageDirector >2014-10-22 16:32:18,383 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 46) Loading extension 'builtin-authn-internal' >2014-10-22 16:32:18,385 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 46) Extension 'builtin-authn-internal' loaded >2014-10-22 16:32:18,388 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 46) Loading extension 'internal' >2014-10-22 16:32:18,391 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 46) Extension 'internal' loaded >2014-10-22 16:32:18,474 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 46) Loading extension 'builtin-authn-qa.lab.tlv.redhat.com' >2014-10-22 16:32:18,483 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 46) Extension 'builtin-authn-qa.lab.tlv.redhat.com' loaded >2014-10-22 16:32:18,544 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 46) Loading extension 'qa.lab.tlv.redhat.com' >2014-10-22 16:32:18,545 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 46) Extension 'qa.lab.tlv.redhat.com' loaded >2014-10-22 16:32:18,546 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 46) Initializing extension 'qa.lab.tlv.redhat.com' >2014-10-22 16:32:18,546 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 46) Extension 'qa.lab.tlv.redhat.com' initialized >2014-10-22 16:32:18,547 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 46) Initializing extension 'builtin-authn-internal' >2014-10-22 16:32:18,548 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 46) Extension 'builtin-authn-internal' initialized >2014-10-22 16:32:18,548 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 46) Initializing extension 'builtin-authn-qa.lab.tlv.redhat.com' >2014-10-22 16:32:18,549 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 46) Extension 'builtin-authn-qa.lab.tlv.redhat.com' initialized >2014-10-22 16:32:18,549 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 46) Initializing extension 'internal' >2014-10-22 16:32:18,549 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 46) Extension 'internal' initialized >2014-10-22 16:32:18,549 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 46) Start of enabled extensions list >2014-10-22 16:32:18,550 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 46) Instance name: 'qa.lab.tlv.redhat.com', Extension name: 'Kerberos/Ldap Authz (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-10-22 16:32:18,550 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 46) Instance name: 'builtin-authn-internal', Extension name: 'Internal Authn (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-10-22 16:32:18,550 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 46) Instance name: 'builtin-authn-qa.lab.tlv.redhat.com', Extension name: 'Kerberos/Ldap Authn (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-10-22 16:32:18,550 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 46) Instance name: 'internal', Extension name: 'Internal Authz (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-10-22 16:32:18,550 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 46) End of enabled extensions list >2014-10-22 16:32:18,843 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (ServerService Thread Pool -- 49) Start initializing DbUserCacheManager >2014-10-22 16:32:18,862 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (ServerService Thread Pool -- 49) Finished initializing DbUserCacheManager >2014-10-22 16:32:18,913 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (ServerService Thread Pool -- 49) Initialization of AsyncTaskManager completed successfully. >2014-10-22 16:32:18,915 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 49) Start initializing ResourceManager >2014-10-22 16:32:19,091 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-1) Principal admin::internal synchronized >2014-10-22 16:32:19,279 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 49) Entered VdsManager constructor >2014-10-22 16:32:19,338 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 49) Initialize vdsBroker (rose05.qa.lab.tlv.redhat.com,54,321) >2014-10-22 16:32:19,589 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 49) VDS 45869c56-4fbd-466b-977a-81604a392b78 was added to the Resource Manager >2014-10-22 16:32:19,589 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 49) Entered VdsManager constructor >2014-10-22 16:32:19,614 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 49) Initialize vdsBroker (master-vds10.qa.lab.tlv.redhat.com,54,321) >2014-10-22 16:32:19,629 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 49) VDS 51fdbad1-2bd5-47d4-aa62-4054e71fbbbf was added to the Resource Manager >2014-10-22 16:32:19,630 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 49) Entered VdsManager constructor >2014-10-22 16:32:19,651 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 49) Initialize vdsBroker (alma06.qa.lab.tlv.redhat.com,54,321) >2014-10-22 16:32:19,704 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 49) VDS 1c39bd88-8672-408f-a593-60d6ef8f647c was added to the Resource Manager >2014-10-22 16:32:19,705 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 49) Entered VdsManager constructor >2014-10-22 16:32:19,732 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 49) Initialize vdsBroker (10.35.70.4,54,321) >2014-10-22 16:32:19,760 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 49) VDS 3a2a45bd-4411-4428-aa45-a514d30d3ea6 was added to the Resource Manager >2014-10-22 16:32:19,801 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 49) Finished initializing ResourceManager >2014-10-22 16:32:19,809 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (ServerService Thread Pool -- 49) Initialization of OvfDataUpdater completed successfully. >2014-10-22 16:32:19,809 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 49) Start scheduling to enable vds load balancer >2014-10-22 16:32:19,824 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 49) Finished scheduling to enable vds load balancer >2014-10-22 16:32:19,824 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 49) Start HA Reservation check >2014-10-22 16:32:19,825 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 49) Finished HA Reservation check >2014-10-22 16:32:19,834 INFO [org.ovirt.engine.core.bll.network.MacPoolManagerRanges] (org.ovirt.thread.pool-7-thread-1) Start initializing MacPoolManagerRanges >2014-10-22 16:32:19,854 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 49) Init VM custom properties utilities >2014-10-22 16:32:19,855 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 49) Init device custom properties utilities >2014-10-22 16:32:19,860 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 49) Initializing Scheduling manager >2014-10-22 16:32:19,938 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 49) Starting external scheduler dicovery thread >2014-10-22 16:32:19,941 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 49) Initialized Scheduling manager >2014-10-22 16:32:19,941 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (ServerService Thread Pool -- 49) Initializing DWH Heart Beat >2014-10-22 16:32:19,942 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (ServerService Thread Pool -- 49) DWH Heart Beat initialized >2014-10-22 16:32:19,946 INFO [org.ovirt.engine.core.bll.network.MacPoolManagerRanges] (org.ovirt.thread.pool-7-thread-1) Finished initializing. Available MACs in pool: 249 >2014-10-22 16:32:20,181 INFO [org.ovirt.engine.core.bll.scheduling.external.ExternalSchedulerDiscoveryThread] (Thread-70) PolicyUnits updated >2014-10-22 16:32:22,739 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-7) Initializing Host: rose05.qa.lab.tlv.redhat.com >2014-10-22 16:32:22,906 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-8) Initializing Host: master-vds10.qa.lab.tlv.redhat.com >2014-10-22 16:32:23,156 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to alma06.qa.lab.tlv.redhat.com/10.35.117.30 >2014-10-22 16:32:23,163 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to /10.35.70.4 >2014-10-22 16:32:24,031 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-10) AuditLogType: UNASSIGNED not exist in string table >2014-10-22 16:32:24,031 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-10) AuditLogType: VDS_HIGH_NETWORK_USE not exist in string table >2014-10-22 16:32:24,032 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-10) AuditLogType: USER_FAILED_REMOVE_VM not exist in string table >2014-10-22 16:32:24,032 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-10) AuditLogType: USER_RUN_UNLOCK_ENTITY_SCRIPT not exist in string table >2014-10-22 16:32:24,033 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-10) AuditLogType: VDS_NETWORK_MTU_DIFFER_FROM_LOGICAL_NETWORK not exist in string table >2014-10-22 16:32:24,034 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-10) AuditLogType: STORAGE_ACTIVATE_ASYNC not exist in string table >2014-10-22 16:32:24,034 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-10) AuditLogType: USER_ADDED_DISK_PROFILE not exist in string table >2014-10-22 16:32:24,034 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-10) AuditLogType: USER_FAILED_TO_ADD_DISK_PROFILE not exist in string table >2014-10-22 16:32:24,034 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-10) AuditLogType: USER_REMOVED_DISK_PROFILE not exist in string table >2014-10-22 16:32:24,034 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-10) AuditLogType: USER_FAILED_TO_REMOVE_DISK_PROFILE not exist in string table >2014-10-22 16:32:24,035 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-10) AuditLogType: USER_UPDATED_DISK_PROFILE not exist in string table >2014-10-22 16:32:24,035 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-10) AuditLogType: USER_FAILED_TO_UPDATE_DISK_PROFILE not exist in string table >2014-10-22 16:32:24,035 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-10) AuditLogType: USER_ADDED_CPU_PROFILE not exist in string table >2014-10-22 16:32:24,035 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-10) AuditLogType: USER_FAILED_TO_ADD_CPU_PROFILE not exist in string table >2014-10-22 16:32:24,035 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-10) AuditLogType: USER_REMOVED_CPU_PROFILE not exist in string table >2014-10-22 16:32:24,035 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-10) AuditLogType: USER_FAILED_TO_REMOVE_CPU_PROFILE not exist in string table >2014-10-22 16:32:24,035 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-10) AuditLogType: USER_UPDATED_CPU_PROFILE not exist in string table >2014-10-22 16:32:24,036 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-10) AuditLogType: USER_FAILED_TO_UPDATE_CPU_PROFILE not exist in string table >2014-10-22 16:32:24,479 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-9) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 16:32:24,488 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-10) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 16:32:24,706 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-9) START, GetHardwareInfoVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c]), log id: 52230952 >2014-10-22 16:32:24,732 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-9) FINISH, GetHardwareInfoVDSCommand, log id: 52230952 >2014-10-22 16:32:25,088 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-10) START, GetHardwareInfoVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, vds=Host[cyan-vdsg.qa.lab.tlv.redhat.com,3a2a45bd-4411-4428-aa45-a514d30d3ea6]), log id: 475cf787 >2014-10-22 16:32:25,138 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-10) FINISH, GetHardwareInfoVDSCommand, log id: 475cf787 >2014-10-22 16:32:25,410 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-10) Initializing Host: cyan-vdsg.qa.lab.tlv.redhat.com >2014-10-22 16:32:25,417 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-9) Initializing Host: alma06.qa.lab.tlv.redhat.com >2014-10-22 16:32:25,718 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-10) [6a4c8f66] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 3a2a45bd-4411-4428-aa45-a514d30d3ea6 Type: VDS >2014-10-22 16:32:25,719 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-9) [1d7a0a03] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 1c39bd88-8672-408f-a593-60d6ef8f647c Type: VDS >2014-10-22 16:32:29,856 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-29) [3a8c61ea] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: f8f5eaee-8fd0-4b45-87db-62d61b03a916 Type: StoragePool >2014-10-22 16:32:29,865 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-29) [3a8c61ea] Storage Pool f8f5eaee-8fd0-4b45-87db-62d61b03a916 - Updating Storage Domain a1341057-9cf8-4750-a734-656d05a5d2c5 status from Active to Unknown, reason : null >2014-10-22 16:32:29,957 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-29) [3a8c61ea] Correlation ID: 3a8c61ea, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center dc_35. Setting status to Non Responsive. >2014-10-22 16:32:30,121 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-29) [3a8c61ea] hostFromVds::selectedVds - alma06.qa.lab.tlv.redhat.com, spmStatus Free, storage pool dc_35 >2014-10-22 16:32:30,133 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-29) [3a8c61ea] Found spm host 3a2a45bd-4411-4428-aa45-a514d30d3ea6, host name: cyan-vdsg.qa.lab.tlv.redhat.com, according to spmId: 5. >2014-10-22 16:32:30,158 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-29) [3a8c61ea] SpmStatus on vds 3a2a45bd-4411-4428-aa45-a514d30d3ea6: SPM >2014-10-22 16:32:30,159 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-29) [3a8c61ea] Using old spm server: cyan-vdsg.qa.lab.tlv.redhat.com, no start needed >2014-10-22 16:32:30,171 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-29) [3a8c61ea] Initialize Irs proxy from vds: 10.35.70.4 >2014-10-22 16:32:30,187 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-29) [3a8c61ea] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Storage Pool Manager runs on Host cyan-vdsg.qa.lab.tlv.redhat.com (Address: 10.35.70.4). >2014-10-22 16:32:30,192 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to /10.35.70.4 >2014-10-22 16:32:30,202 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-11) [3a8c61ea] START, SPMGetAllTasksInfoVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false), log id: 440c19a2 >2014-10-22 16:32:30,396 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-11) [3a8c61ea] -- executeIrsBrokerCommand: Attempting on storage pool f8f5eaee-8fd0-4b45-87db-62d61b03a916 >2014-10-22 16:32:30,407 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-11) [3a8c61ea] START, HSMGetAllTasksInfoVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6), log id: 7f543f58 >2014-10-22 16:32:30,419 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-11) [3a8c61ea] FINISH, HSMGetAllTasksInfoVDSCommand, return: [], log id: 7f543f58 >2014-10-22 16:32:30,419 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-11) [3a8c61ea] FINISH, SPMGetAllTasksInfoVDSCommand, return: [], log id: 440c19a2 >2014-10-22 16:32:30,420 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-7-thread-11) [3a8c61ea] Discovered no tasks on Storage Pool dc_35 >2014-10-22 16:32:37,877 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-2) Running command: LoginUserCommand internal: false. >2014-10-22 16:32:37,900 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-10-22 16:39:00,879 INFO [org.ovirt.engine.ui.frontend.server.gwt.plugin.PluginDataManager] (ajp-/127.0.0.1:8702-6) Reading UI plugin descriptor [/usr/share/ovirt-engine/ui-plugins/redhat_support_plugin_rhev.json] >2014-10-22 16:39:00,881 INFO [org.ovirt.engine.ui.frontend.server.gwt.plugin.PluginDataManager] (ajp-/127.0.0.1:8702-6) Reading UI plugin configuration [/etc/ovirt-engine/ui-plugins/redhat_support_plugin_rhev-config.json] >2014-10-22 16:39:00,949 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-6) Invalid locale found in configuration: >2014-10-22 16:39:00,950 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-6) Invalid locale found in configuration: >2014-10-22 16:39:06,255 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (ajp-/127.0.0.1:8702-5) Running command: LoginAdminUserCommand internal: false. >2014-10-22 16:39:06,273 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-10-22 16:39:25,162 INFO [org.ovirt.engine.core.bll.AddVmFromScratchCommand] (ajp-/127.0.0.1:8702-6) [538119ed] Lock Acquired to object EngineLock [exclusiveLocks= key: baloon_vm value: VM_NAME >, sharedLocks= ] >2014-10-22 16:39:25,238 INFO [org.ovirt.engine.core.bll.AddVmFromScratchCommand] (ajp-/127.0.0.1:8702-6) [538119ed] Running command: AddVmFromScratchCommand internal: false. Entities affected : ID: 08ea0169-ae42-4e1b-bb20-839a66cd2010 Type: VdsGroupsAction group CREATE_VM with role type USER >2014-10-22 16:39:25,572 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) [538119ed] Correlation ID: 538119ed, Job ID: 84188456-4820-4a7e-9500-301567e4dc9c, Call Stack: null, Custom Event ID: -1, Message: VM baloon_vm was created by admin. >2014-10-22 16:39:25,589 INFO [org.ovirt.engine.core.bll.AddVmFromScratchCommand] (ajp-/127.0.0.1:8702-6) [538119ed] Lock freed to object EngineLock [exclusiveLocks= key: baloon_vm value: VM_NAME >, sharedLocks= ] >2014-10-22 16:39:27,720 INFO [org.ovirt.engine.core.bll.network.vm.AddVmInterfaceCommand] (ajp-/127.0.0.1:8702-5) [240d28ff] Running command: AddVmInterfaceCommand internal: false. Entities affected : ID: 3e13f009-03f3-4a43-b540-ad7fe3122e53 Type: VMAction group CONFIGURE_VM_NETWORK with role type USER, ID: c078ca43-dc90-4ffe-a17e-68721438ee95 Type: VnicProfileAction group CONFIGURE_VM_NETWORK with role type USER >2014-10-22 16:39:28,045 INFO [org.ovirt.engine.core.bll.network.vm.ActivateDeactivateVmNicCommand] (ajp-/127.0.0.1:8702-5) [170995b0] Running command: ActivateDeactivateVmNicCommand internal: true. Entities affected : ID: 3e13f009-03f3-4a43-b540-ad7fe3122e53 Type: VM >2014-10-22 16:39:28,145 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [170995b0] Correlation ID: 170995b0, Call Stack: null, Custom Event ID: -1, Message: Network Interface nic1 (VirtIO) was plugged to VM baloon_vm. (User: admin) >2014-10-22 16:39:28,196 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [170995b0] Correlation ID: 240d28ff, Call Stack: null, Custom Event ID: -1, Message: Interface nic1 (VirtIO) was added to VM baloon_vm. (User: admin) >2014-10-22 16:39:28,576 INFO [org.ovirt.engine.core.bll.network.vm.ReorderVmNicsCommand] (ajp-/127.0.0.1:8702-2) [7fee8817] Running command: ReorderVmNicsCommand internal: false. Entities affected : ID: 3e13f009-03f3-4a43-b540-ad7fe3122e53 Type: VMAction group CREATE_VM with role type USER >2014-10-22 16:39:30,888 INFO [org.ovirt.engine.core.bll.AddDiskCommand] (ajp-/127.0.0.1:8702-2) [6efbfa95] Lock Acquired to object EngineLock [exclusiveLocks= key: 3e13f009-03f3-4a43-b540-ad7fe3122e53 value: VM_DISK_BOOT >, sharedLocks= key: 3e13f009-03f3-4a43-b540-ad7fe3122e53 value: VM >] >2014-10-22 16:39:30,984 INFO [org.ovirt.engine.core.bll.AddDiskCommand] (ajp-/127.0.0.1:8702-2) [6efbfa95] Running command: AddDiskCommand internal: false. Entities affected : ID: 3e13f009-03f3-4a43-b540-ad7fe3122e53 Type: VMAction group CONFIGURE_VM_STORAGE with role type USER, ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: StorageAction group CREATE_DISK with role type USER >2014-10-22 16:39:31,374 INFO [org.ovirt.engine.core.bll.AddImageFromScratchCommand] (ajp-/127.0.0.1:8702-2) [1cee0ee6] Running command: AddImageFromScratchCommand internal: true. Entities affected : ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: Storage >2014-10-22 16:39:31,469 INFO [org.ovirt.engine.core.bll.AddImageFromScratchCommand] (ajp-/127.0.0.1:8702-2) [1cee0ee6] Lock freed to object EngineLock [exclusiveLocks= key: 3e13f009-03f3-4a43-b540-ad7fe3122e53 value: VM_DISK_BOOT >, sharedLocks= key: 3e13f009-03f3-4a43-b540-ad7fe3122e53 value: VM >] >2014-10-22 16:39:31,472 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (ajp-/127.0.0.1:8702-2) [1cee0ee6] START, CreateImageVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = e3162e04-764d-40f0-993c-34865796c725, imageSizeInBytes = 1073741824, volumeFormat = RAW, newImageId = 4e6c4c48-d697-46ce-9fe2-3782b0516b73, newImageDescription = ), log id: 6aa7bcb7 >2014-10-22 16:39:31,475 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (ajp-/127.0.0.1:8702-2) [1cee0ee6] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-10-22 16:39:31,642 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (ajp-/127.0.0.1:8702-2) [1cee0ee6] FINISH, CreateImageVDSCommand, return: 4e6c4c48-d697-46ce-9fe2-3782b0516b73, log id: 6aa7bcb7 >2014-10-22 16:39:31,693 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (ajp-/127.0.0.1:8702-2) [1cee0ee6] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command ef809ccf-9893-4c29-8e94-e9d333014bd0 >2014-10-22 16:39:31,693 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (ajp-/127.0.0.1:8702-2) [1cee0ee6] CommandMultiAsyncTasks::AttachTask: Attaching task 44a4e52a-4d39-4b8c-8f25-b9f5cf721499 to command ef809ccf-9893-4c29-8e94-e9d333014bd0. >2014-10-22 16:39:31,762 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (ajp-/127.0.0.1:8702-2) [1cee0ee6] Adding task 44a4e52a-4d39-4b8c-8f25-b9f5cf721499 (Parent Command AddDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-10-22 16:39:31,848 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) [1cee0ee6] Correlation ID: 6efbfa95, Job ID: 0b0fb79d-16d8-44f3-b52f-ff0623f3c61c, Call Stack: null, Custom Event ID: -1, Message: Add-Disk operation of baloon_vm_Disk1 was initiated on VM baloon_vm by admin. >2014-10-22 16:39:31,849 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (ajp-/127.0.0.1:8702-2) [1cee0ee6] BaseAsyncTask::startPollingTask: Starting to poll task 44a4e52a-4d39-4b8c-8f25-b9f5cf721499. >2014-10-22 16:39:39,019 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-99) Polling and updating Async Tasks: 1 tasks, 1 tasks to poll now >2014-10-22 16:39:39,038 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-99) SPMAsyncTask::PollTask: Polling task 44a4e52a-4d39-4b8c-8f25-b9f5cf721499 (Parent Command AddDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-10-22 16:39:39,049 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-99) BaseAsyncTask::onTaskEndSuccess: Task 44a4e52a-4d39-4b8c-8f25-b9f5cf721499 (Parent Command AddDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-10-22 16:39:39,050 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-99) CommandAsyncTask::endActionIfNecessary: All tasks of command ef809ccf-9893-4c29-8e94-e9d333014bd0 has ended -> executing endAction >2014-10-22 16:39:39,052 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-99) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: ef809ccf-9893-4c29-8e94-e9d333014bd0): calling endAction . >2014-10-22 16:39:39,053 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-4) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction AddDisk, executionIndex: 0 >2014-10-22 16:39:39,085 INFO [org.ovirt.engine.core.bll.AddDiskCommand] (org.ovirt.thread.pool-7-thread-4) [6efbfa95] Ending command successfully: org.ovirt.engine.core.bll.AddDiskCommand >2014-10-22 16:39:39,106 INFO [org.ovirt.engine.core.bll.AddImageFromScratchCommand] (org.ovirt.thread.pool-7-thread-4) [1cee0ee6] Ending command successfully: org.ovirt.engine.core.bll.AddImageFromScratchCommand >2014-10-22 16:39:39,129 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-4) [1cee0ee6] START, GetImageInfoVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = e3162e04-764d-40f0-993c-34865796c725, imageId = 4e6c4c48-d697-46ce-9fe2-3782b0516b73), log id: 7846ba6c >2014-10-22 16:39:39,179 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-4) [1cee0ee6] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@626bc7e3, log id: 7846ba6c >2014-10-22 16:39:39,214 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-4) Correlation ID: 6efbfa95, Job ID: 0b0fb79d-16d8-44f3-b52f-ff0623f3c61c, Call Stack: null, Custom Event ID: -1, Message: The disk baloon_vm_Disk1 was successfully added to VM baloon_vm. >2014-10-22 16:39:39,214 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-4) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddDisk completed, handling the result. >2014-10-22 16:39:39,214 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-4) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddDisk succeeded, clearing tasks. >2014-10-22 16:39:39,219 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-4) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 44a4e52a-4d39-4b8c-8f25-b9f5cf721499 >2014-10-22 16:39:39,220 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-4) START, SPMClearTaskVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, taskId = 44a4e52a-4d39-4b8c-8f25-b9f5cf721499), log id: 76993433 >2014-10-22 16:39:39,224 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-4) START, HSMClearTaskVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, taskId=44a4e52a-4d39-4b8c-8f25-b9f5cf721499), log id: 4ed2dbfb >2014-10-22 16:39:39,274 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-4) FINISH, HSMClearTaskVDSCommand, log id: 4ed2dbfb >2014-10-22 16:39:39,275 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-4) FINISH, SPMClearTaskVDSCommand, log id: 76993433 >2014-10-22 16:39:39,293 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-4) BaseAsyncTask::removeTaskFromDB: Removed task 44a4e52a-4d39-4b8c-8f25-b9f5cf721499 from DataBase >2014-10-22 16:39:39,293 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-4) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity ef809ccf-9893-4c29-8e94-e9d333014bd0 >2014-10-22 16:40:48,896 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-39) Setting new tasks map. The map contains now 0 tasks >2014-10-22 16:40:48,896 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-39) Cleared all tasks of pool f8f5eaee-8fd0-4b45-87db-62d61b03a916. >2014-10-22 16:41:28,477 INFO [org.ovirt.engine.core.bll.storage.AddStorageServerConnectionCommand] (ajp-/127.0.0.1:8702-6) [2abd16c0] Lock Acquired to object EngineLock [exclusiveLocks= key: 10.35.160.108:/RHEV/artyom_export_domain value: STORAGE_CONNECTION >, sharedLocks= ] >2014-10-22 16:41:28,507 INFO [org.ovirt.engine.core.bll.storage.AddStorageServerConnectionCommand] (ajp-/127.0.0.1:8702-6) [2abd16c0] Running command: AddStorageServerConnectionCommand internal: false. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-10-22 16:41:28,512 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (ajp-/127.0.0.1:8702-6) [2abd16c0] START, ConnectStorageServerVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: null, connection: 10.35.160.108:/RHEV/artyom_export_domain, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 76b17d9c >2014-10-22 16:41:28,745 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (ajp-/127.0.0.1:8702-6) [2abd16c0] FINISH, ConnectStorageServerVDSCommand, return: {00000000-0000-0000-0000-000000000000=0}, log id: 76b17d9c >2014-10-22 16:41:28,763 INFO [org.ovirt.engine.core.bll.storage.AddStorageServerConnectionCommand] (ajp-/127.0.0.1:8702-6) [2abd16c0] Lock freed to object EngineLock [exclusiveLocks= key: 10.35.160.108:/RHEV/artyom_export_domain value: STORAGE_CONNECTION >, sharedLocks= ] >2014-10-22 16:41:28,813 INFO [org.ovirt.engine.core.bll.storage.GetExistingStorageDomainListQuery] (ajp-/127.0.0.1:8702-4) START, GetExistingStorageDomainListQuery(refresh: true, filtered: false), log id: 64850235 >2014-10-22 16:41:28,819 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetStorageDomainsListVDSCommand] (ajp-/127.0.0.1:8702-4) START, HSMGetStorageDomainsListVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, storagePoolId=00000000-0000-0000-0000-000000000000, storageType=NFS, storageDomainType=ImportExport, path=10.35.160.108:/RHEV/artyom_export_domain), log id: 38655465 >2014-10-22 16:41:29,631 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetStorageDomainsListVDSCommand] (ajp-/127.0.0.1:8702-4) FINISH, HSMGetStorageDomainsListVDSCommand, return: [45be96a3-8ccc-4807-8bd6-ff4960c451aa], log id: 38655465 >2014-10-22 16:41:29,747 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetStorageDomainInfoVDSCommand] (ajp-/127.0.0.1:8702-4) START, HSMGetStorageDomainInfoVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, storageDomainId=45be96a3-8ccc-4807-8bd6-ff4960c451aa), log id: 19649051 >2014-10-22 16:41:29,768 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetStorageDomainInfoVDSCommand] (ajp-/127.0.0.1:8702-4) FINISH, HSMGetStorageDomainInfoVDSCommand, return: <StorageDomainStatic[export_domain, 45be96a3-8ccc-4807-8bd6-ff4960c451aa], ac1052a2-aee4-40ed-bcce-9ebd9242170f>, log id: 19649051 >2014-10-22 16:41:29,769 INFO [org.ovirt.engine.core.bll.storage.GetExistingStorageDomainListQuery] (ajp-/127.0.0.1:8702-4) FINISH, GetExistingStorageDomainListQuery, log id: 64850235 >2014-10-22 16:41:30,220 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetStorageDomainsListVDSCommand] (ajp-/127.0.0.1:8702-2) [3367090d] START, HSMGetStorageDomainsListVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, storagePoolId=00000000-0000-0000-0000-000000000000, storageType=NFS, storageDomainType=ImportExport, path=), log id: 2b9fc63d >2014-10-22 16:41:30,804 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetStorageDomainsListVDSCommand] (ajp-/127.0.0.1:8702-2) [3367090d] FINISH, HSMGetStorageDomainsListVDSCommand, return: [45be96a3-8ccc-4807-8bd6-ff4960c451aa], log id: 2b9fc63d >2014-10-22 16:41:30,841 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetStorageDomainInfoVDSCommand] (ajp-/127.0.0.1:8702-2) [3367090d] START, HSMGetStorageDomainInfoVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, storageDomainId=45be96a3-8ccc-4807-8bd6-ff4960c451aa), log id: 1b3bbd1e >2014-10-22 16:41:30,878 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetStorageDomainInfoVDSCommand] (ajp-/127.0.0.1:8702-2) [3367090d] FINISH, HSMGetStorageDomainInfoVDSCommand, return: <StorageDomainStatic[export_domain, 45be96a3-8ccc-4807-8bd6-ff4960c451aa], ac1052a2-aee4-40ed-bcce-9ebd9242170f>, log id: 1b3bbd1e >2014-10-22 16:41:30,948 INFO [org.ovirt.engine.core.bll.storage.AddExistingFileStorageDomainCommand] (ajp-/127.0.0.1:8702-2) [3367090d] Running command: AddExistingFileStorageDomainCommand internal: false. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-10-22 16:41:30,990 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStorageDomainStatsVDSCommand] (ajp-/127.0.0.1:8702-2) [3367090d] START, GetStorageDomainStatsVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, storageDomainId=45be96a3-8ccc-4807-8bd6-ff4960c451aa), log id: 6ded00b3 >2014-10-22 16:41:31,008 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStorageDomainStatsVDSCommand] (ajp-/127.0.0.1:8702-2) [3367090d] FINISH, GetStorageDomainStatsVDSCommand, return: StorageDomain[, 45be96a3-8ccc-4807-8bd6-ff4960c451aa], log id: 6ded00b3 >2014-10-22 16:41:31,098 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) [3367090d] Correlation ID: 3367090d, Job ID: 09f8558f-a5b2-454e-968e-fa34ce9dab7d, Call Stack: null, Custom Event ID: -1, Message: Storage Domain export_domain was added by admin >2014-10-22 16:41:31,202 INFO [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] (ajp-/127.0.0.1:8702-5) [448b9842] Lock Acquired to object EngineLock [exclusiveLocks= key: 45be96a3-8ccc-4807-8bd6-ff4960c451aa value: STORAGE >, sharedLocks= ] >2014-10-22 16:41:31,231 INFO [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] (ajp-/127.0.0.1:8702-5) [448b9842] Running command: AttachStorageDomainToPoolCommand internal: false. Entities affected : ID: 45be96a3-8ccc-4807-8bd6-ff4960c451aa Type: StorageAction group MANIPULATE_STORAGE_DOMAIN with role type ADMIN >2014-10-22 16:41:31,264 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (org.ovirt.thread.pool-7-thread-9) [13353459] Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-10-22 16:41:31,268 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-9) [13353459] START, ConnectStorageServerVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: dbd0d3f7-7879-41fc-baf6-afd118622364, connection: 10.35.160.108:/RHEV/artyom_export_domain, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 6f10478f >2014-10-22 16:41:31,270 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (org.ovirt.thread.pool-7-thread-13) [8151fae] Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-10-22 16:41:31,281 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-13) [8151fae] START, ConnectStorageServerVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: dbd0d3f7-7879-41fc-baf6-afd118622364, connection: 10.35.160.108:/RHEV/artyom_export_domain, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 21beca9 >2014-10-22 16:41:31,306 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-13) [8151fae] FINISH, ConnectStorageServerVDSCommand, return: {dbd0d3f7-7879-41fc-baf6-afd118622364=0}, log id: 21beca9 >2014-10-22 16:41:31,393 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-9) [13353459] FINISH, ConnectStorageServerVDSCommand, return: {dbd0d3f7-7879-41fc-baf6-afd118622364=0}, log id: 6f10478f >2014-10-22 16:41:31,396 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand] (ajp-/127.0.0.1:8702-5) [448b9842] START, AttachStorageDomainVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = 45be96a3-8ccc-4807-8bd6-ff4960c451aa), log id: 3cdbbc9e >2014-10-22 16:41:53,992 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand] (ajp-/127.0.0.1:8702-5) [448b9842] Command AttachStorageDomainVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = 45be96a3-8ccc-4807-8bd6-ff4960c451aa) execution failed. Exception: IrsOperationFailedNoFailoverException: IRSGenericException: IRSErrorException: Storage domain already attached to pool: u'domain=45be96a3-8ccc-4807-8bd6-ff4960c451aa, pool=ac1052a2-aee4-40ed-bcce-9ebd9242170f' >2014-10-22 16:41:53,993 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand] (ajp-/127.0.0.1:8702-5) [448b9842] FINISH, AttachStorageDomainVDSCommand, log id: 3cdbbc9e >2014-10-22 16:41:53,993 ERROR [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] (ajp-/127.0.0.1:8702-5) [448b9842] Command org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand throw Vdc Bll exception. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.irsbroker.IrsOperationFailedNoFailoverException: IRSGenericException: IRSErrorException: Storage domain already attached to pool: u'domain=45be96a3-8ccc-4807-8bd6-ff4960c451aa, pool=ac1052a2-aee4-40ed-bcce-9ebd9242170f' (Failed with error StorageDomainAlreadyAttached and code 380) >2014-10-22 16:41:54,021 INFO [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] (ajp-/127.0.0.1:8702-5) [448b9842] Command [id=42bb6c80-66bd-4835-857f-81984d0621aa]: Compensating NEW_ENTITY_ID of org.ovirt.engine.core.common.businessentities.StoragePoolIsoMap; snapshot: storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, storageId = 45be96a3-8ccc-4807-8bd6-ff4960c451aa. >2014-10-22 16:41:54,060 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [448b9842] Correlation ID: 448b9842, Job ID: 590e1388-b711-4629-b5af-b3a0ecb2c7d4, Call Stack: null, Custom Event ID: -1, Message: Failed to attach Storage Domain export_domain to Data Center dc_35. (User: admin) >2014-10-22 16:41:54,083 INFO [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] (ajp-/127.0.0.1:8702-5) [448b9842] Lock freed to object EngineLock [exclusiveLocks= key: 45be96a3-8ccc-4807-8bd6-ff4960c451aa value: STORAGE >, sharedLocks= ] >2014-10-22 16:43:14,595 INFO [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] (ajp-/127.0.0.1:8702-7) [6870c241] Lock Acquired to object EngineLock [exclusiveLocks= key: 45be96a3-8ccc-4807-8bd6-ff4960c451aa value: STORAGE >, sharedLocks= ] >2014-10-22 16:43:14,639 INFO [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] (org.ovirt.thread.pool-7-thread-12) [6870c241] Running command: AttachStorageDomainToPoolCommand internal: false. Entities affected : ID: 45be96a3-8ccc-4807-8bd6-ff4960c451aa Type: StorageAction group MANIPULATE_STORAGE_DOMAIN with role type ADMIN >2014-10-22 16:43:14,653 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (org.ovirt.thread.pool-7-thread-13) [3bc76d4] Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-10-22 16:43:14,653 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (org.ovirt.thread.pool-7-thread-9) [4a1d5a36] Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-10-22 16:43:14,656 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-13) [3bc76d4] START, ConnectStorageServerVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: dbd0d3f7-7879-41fc-baf6-afd118622364, connection: 10.35.160.108:/RHEV/artyom_export_domain, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 7b3ed9f9 >2014-10-22 16:43:14,666 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-9) [4a1d5a36] START, ConnectStorageServerVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: dbd0d3f7-7879-41fc-baf6-afd118622364, connection: 10.35.160.108:/RHEV/artyom_export_domain, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 1916035 >2014-10-22 16:43:14,681 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-13) [3bc76d4] FINISH, ConnectStorageServerVDSCommand, return: {dbd0d3f7-7879-41fc-baf6-afd118622364=0}, log id: 7b3ed9f9 >2014-10-22 16:43:14,695 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-9) [4a1d5a36] FINISH, ConnectStorageServerVDSCommand, return: {dbd0d3f7-7879-41fc-baf6-afd118622364=0}, log id: 1916035 >2014-10-22 16:43:14,697 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand] (org.ovirt.thread.pool-7-thread-12) [6870c241] START, AttachStorageDomainVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = 45be96a3-8ccc-4807-8bd6-ff4960c451aa), log id: 489db0e4 >2014-10-22 16:43:36,343 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand] (org.ovirt.thread.pool-7-thread-12) [6870c241] FINISH, AttachStorageDomainVDSCommand, log id: 489db0e4 >2014-10-22 16:43:36,397 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImagesListVDSCommand] (org.ovirt.thread.pool-7-thread-12) [6870c241] START, GetImagesListVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, sdUUID = 45be96a3-8ccc-4807-8bd6-ff4960c451aa), log id: 2fbc7ade >2014-10-22 16:43:36,440 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImagesListVDSCommand] (org.ovirt.thread.pool-7-thread-12) [6870c241] FINISH, GetImagesListVDSCommand, return: [d733fe32-f0c8-4966-8757-aadd002fe374, dc697da2-0667-4125-b37a-0b0506c6555a, 9342a75c-674d-4d60-b53e-3007782e0c69], log id: 2fbc7ade >2014-10-22 16:43:36,492 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVolumesListVDSCommand] (org.ovirt.thread.pool-7-thread-12) [6870c241] START, GetVolumesListVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = 45be96a3-8ccc-4807-8bd6-ff4960c451aa, imageGroupId = d733fe32-f0c8-4966-8757-aadd002fe374), log id: 6e4c2b5b >2014-10-22 16:43:36,510 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVolumesListVDSCommand] (org.ovirt.thread.pool-7-thread-12) [6870c241] FINISH, GetVolumesListVDSCommand, return: [be2b2695-051b-4d73-b4ca-90b10e9467ee], log id: 6e4c2b5b >2014-10-22 16:43:36,511 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-12) [6870c241] START, GetImageInfoVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = 45be96a3-8ccc-4807-8bd6-ff4960c451aa, imageGroupId = d733fe32-f0c8-4966-8757-aadd002fe374, imageId = be2b2695-051b-4d73-b4ca-90b10e9467ee), log id: 4260c14d >2014-10-22 16:43:36,558 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-12) [6870c241] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@49803893, log id: 4260c14d >2014-10-22 16:43:36,600 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVolumesListVDSCommand] (org.ovirt.thread.pool-7-thread-12) [6870c241] START, GetVolumesListVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = 45be96a3-8ccc-4807-8bd6-ff4960c451aa, imageGroupId = dc697da2-0667-4125-b37a-0b0506c6555a), log id: 447c39e2 >2014-10-22 16:43:36,616 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVolumesListVDSCommand] (org.ovirt.thread.pool-7-thread-12) [6870c241] FINISH, GetVolumesListVDSCommand, return: [e79073ae-e3d6-4410-87b4-787c1884a214], log id: 447c39e2 >2014-10-22 16:43:36,616 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-12) [6870c241] START, GetImageInfoVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = 45be96a3-8ccc-4807-8bd6-ff4960c451aa, imageGroupId = dc697da2-0667-4125-b37a-0b0506c6555a, imageId = e79073ae-e3d6-4410-87b4-787c1884a214), log id: 37bc478e >2014-10-22 16:43:36,657 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-12) [6870c241] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@b7f7b79b, log id: 37bc478e >2014-10-22 16:43:36,662 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVolumesListVDSCommand] (org.ovirt.thread.pool-7-thread-12) [6870c241] START, GetVolumesListVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = 45be96a3-8ccc-4807-8bd6-ff4960c451aa, imageGroupId = 9342a75c-674d-4d60-b53e-3007782e0c69), log id: 218efeae >2014-10-22 16:43:36,688 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVolumesListVDSCommand] (org.ovirt.thread.pool-7-thread-12) [6870c241] FINISH, GetVolumesListVDSCommand, return: [9d6f64ba-3481-4e31-997e-7b745fb610ab], log id: 218efeae >2014-10-22 16:43:36,688 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-12) [6870c241] START, GetImageInfoVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = 45be96a3-8ccc-4807-8bd6-ff4960c451aa, imageGroupId = 9342a75c-674d-4d60-b53e-3007782e0c69, imageId = 9d6f64ba-3481-4e31-997e-7b745fb610ab), log id: 32f262b8 >2014-10-22 16:43:36,727 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-12) [6870c241] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@9621f3a3, log id: 32f262b8 >2014-10-22 16:43:36,728 WARN [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] (org.ovirt.thread.pool-7-thread-12) There are no OVF_STORE disks on storage domain id 45be96a3-8ccc-4807-8bd6-ff4960c451aa >2014-10-22 16:43:36,749 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-7-thread-12) [2f42ee33] Running command: ActivateStorageDomainCommand internal: true. Entities affected : ID: 45be96a3-8ccc-4807-8bd6-ff4960c451aa Type: StorageAction group MANIPULATE_STORAGE_DOMAIN with role type ADMIN >2014-10-22 16:43:36,757 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-7-thread-12) [2f42ee33] Lock freed to object EngineLock [exclusiveLocks= key: 45be96a3-8ccc-4807-8bd6-ff4960c451aa value: STORAGE >, sharedLocks= ] >2014-10-22 16:43:36,758 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-7-thread-12) [2f42ee33] ActivateStorage Domain. Before Connect all hosts to pool. Time:10/22/14 4:43 PM >2014-10-22 16:43:36,764 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (org.ovirt.thread.pool-7-thread-30) [8eea8a1] Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-10-22 16:43:36,767 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (org.ovirt.thread.pool-7-thread-38) [334ffa82] Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-10-22 16:43:36,777 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-30) [8eea8a1] START, ConnectStorageServerVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: dbd0d3f7-7879-41fc-baf6-afd118622364, connection: 10.35.160.108:/RHEV/artyom_export_domain, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 1e48ac63 >2014-10-22 16:43:36,780 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-38) [334ffa82] START, ConnectStorageServerVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: dbd0d3f7-7879-41fc-baf6-afd118622364, connection: 10.35.160.108:/RHEV/artyom_export_domain, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 352da71d >2014-10-22 16:43:36,789 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-30) [8eea8a1] FINISH, ConnectStorageServerVDSCommand, return: {dbd0d3f7-7879-41fc-baf6-afd118622364=0}, log id: 1e48ac63 >2014-10-22 16:43:36,794 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-38) [334ffa82] FINISH, ConnectStorageServerVDSCommand, return: {dbd0d3f7-7879-41fc-baf6-afd118622364=0}, log id: 352da71d >2014-10-22 16:43:36,798 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (org.ovirt.thread.pool-7-thread-12) [2f42ee33] START, ActivateStorageDomainVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = 45be96a3-8ccc-4807-8bd6-ff4960c451aa), log id: 303187bb >2014-10-22 16:43:36,998 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (org.ovirt.thread.pool-7-thread-12) [2f42ee33] FINISH, ActivateStorageDomainVDSCommand, return: true, log id: 303187bb >2014-10-22 16:43:36,998 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-7-thread-12) [2f42ee33] ActivateStorage Domain. After Connect all hosts to pool. Time:10/22/14 4:43 PM >2014-10-22 16:43:37,017 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-7-thread-41) START, ConnectStoragePoolVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, vdsId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, masterVersion = 1), log id: 12495fab >2014-10-22 16:43:37,017 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-7-thread-36) START, ConnectStoragePoolVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vdsId = 1c39bd88-8672-408f-a593-60d6ef8f647c, storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, masterVersion = 1), log id: 2186b2df >2014-10-22 16:43:37,377 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-7-thread-36) FINISH, ConnectStoragePoolVDSCommand, log id: 2186b2df >2014-10-22 16:43:37,377 INFO [org.ovirt.engine.core.bll.storage.RefreshPoolSingleAsyncOperation] (org.ovirt.thread.pool-7-thread-36) Refreshed vds alma06.qa.lab.tlv.redhat.com in pool dc_35 >2014-10-22 16:43:37,691 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-7-thread-41) FINISH, ConnectStoragePoolVDSCommand, log id: 12495fab >2014-10-22 16:43:37,691 INFO [org.ovirt.engine.core.bll.storage.RefreshPoolSingleAsyncOperation] (org.ovirt.thread.pool-7-thread-41) Refreshed vds cyan-vdsg.qa.lab.tlv.redhat.com in pool dc_35 >2014-10-22 16:43:37,693 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-7-thread-12) [2f42ee33] ActivateStorage Domain. After change storage pool status in vds. Time:10/22/14 4:43 PM >2014-10-22 16:43:37,713 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-12) [2f42ee33] Correlation ID: 2f42ee33, Call Stack: null, Custom Event ID: -1, Message: Storage Domain export_domain (Data Center dc_35) was activated by admin >2014-10-22 16:43:37,733 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-12) [2f42ee33] Correlation ID: 6870c241, Job ID: ef623f58-36be-4643-863e-5ecde866ee7e, Call Stack: null, Custom Event ID: -1, Message: Storage Domain export_domain was attached to Data Center dc_35 by admin >2014-10-22 16:43:37,749 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-12) [2f42ee33] Trying to release exclusive lock which does not exist, lock key: 45be96a3-8ccc-4807-8bd6-ff4960c451aaSTORAGE >2014-10-22 16:43:37,750 INFO [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] (org.ovirt.thread.pool-7-thread-12) [2f42ee33] Lock freed to object EngineLock [exclusiveLocks= key: 45be96a3-8ccc-4807-8bd6-ff4960c451aa value: STORAGE >, sharedLocks= ] >2014-10-22 16:43:41,022 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVmsInfoVDSCommand] (ajp-/127.0.0.1:8702-8) START, GetVmsInfoVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = 45be96a3-8ccc-4807-8bd6-ff4960c451aa, vmIdList = null), log id: 65760a8a >2014-10-22 16:43:41,121 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVmsInfoVDSCommand] (ajp-/127.0.0.1:8702-8) FINISH, GetVmsInfoVDSCommand, log id: 65760a8a >2014-10-22 16:43:55,239 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-9) Running command: LoginUserCommand internal: false. >2014-10-22 16:43:55,244 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-10-22 16:43:56,183 INFO [org.ovirt.engine.core.bll.aaa.LogoutBySessionCommand] (ajp-/127.0.0.1:8702-9) [74b8bcb1] Running command: LogoutBySessionCommand internal: false. >2014-10-22 16:43:56,214 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-9) [51ecae0b] Running command: LogoutUserCommand internal: false. >2014-10-22 16:43:56,234 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [51ecae0b] Correlation ID: 51ecae0b, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-10-22 16:44:06,497 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-4) Running command: LoginUserCommand internal: false. >2014-10-22 16:44:06,502 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-10-22 16:44:19,440 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-8) Running command: LoginUserCommand internal: false. >2014-10-22 16:44:19,446 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-8) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-10-22 16:44:20,310 INFO [org.ovirt.engine.core.bll.aaa.LogoutBySessionCommand] (ajp-/127.0.0.1:8702-8) [45a95502] Running command: LogoutBySessionCommand internal: false. >2014-10-22 16:44:20,314 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-8) [5e24bbd3] Running command: LogoutUserCommand internal: false. >2014-10-22 16:44:20,318 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-8) [5e24bbd3] Correlation ID: 5e24bbd3, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-10-22 16:44:51,129 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-9) Running command: LoginUserCommand internal: false. >2014-10-22 16:44:51,134 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-10-22 16:44:51,222 INFO [org.ovirt.engine.core.bll.aaa.LogoutBySessionCommand] (ajp-/127.0.0.1:8702-9) [c5ac40c] Running command: LogoutBySessionCommand internal: false. >2014-10-22 16:44:51,224 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-9) [6224dca5] Running command: LogoutUserCommand internal: false. >2014-10-22 16:44:51,227 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [6224dca5] Correlation ID: 6224dca5, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-10-22 16:44:58,969 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-4) Running command: LoginUserCommand internal: false. >2014-10-22 16:44:58,974 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-10-22 16:44:59,100 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp-/127.0.0.1:8702-4) [11e2d426] Lock Acquired to object EngineLock [exclusiveLocks= key: baloon_vm value: VM_NAME >, sharedLocks= key: 3e13f009-03f3-4a43-b540-ad7fe3122e53 value: VM >] >2014-10-22 16:44:59,206 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp-/127.0.0.1:8702-4) [11e2d426] Running command: UpdateVmCommand internal: false. Entities affected : ID: 3e13f009-03f3-4a43-b540-ad7fe3122e53 Type: VMAction group EDIT_VM_PROPERTIES with role type USER >2014-10-22 16:44:59,259 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [11e2d426] Correlation ID: 11e2d426, Job ID: f82a2030-a499-4da2-b68f-4dd804e4ca1b, Call Stack: null, Custom Event ID: -1, Message: VM baloon_vm configuration was updated by admin. >2014-10-22 16:44:59,264 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp-/127.0.0.1:8702-4) [11e2d426] Lock freed to object EngineLock [exclusiveLocks= key: baloon_vm value: VM_NAME >, sharedLocks= key: 3e13f009-03f3-4a43-b540-ad7fe3122e53 value: VM >] >2014-10-22 16:44:59,489 INFO [org.ovirt.engine.core.bll.aaa.LogoutBySessionCommand] (ajp-/127.0.0.1:8702-4) [286d2d63] Running command: LogoutBySessionCommand internal: false. >2014-10-22 16:44:59,504 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-4) [62211509] Running command: LogoutUserCommand internal: false. >2014-10-22 16:44:59,508 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [62211509] Correlation ID: 62211509, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-10-22 16:45:06,515 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-7) Running command: LoginUserCommand internal: false. >2014-10-22 16:45:06,523 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-10-22 16:45:22,958 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVmsInfoVDSCommand] (ajp-/127.0.0.1:8702-1) START, GetVmsInfoVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = 45be96a3-8ccc-4807-8bd6-ff4960c451aa, vmIdList = null), log id: 6ce67701 >2014-10-22 16:45:23,000 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVmsInfoVDSCommand] (ajp-/127.0.0.1:8702-1) FINISH, GetVmsInfoVDSCommand, log id: 6ce67701 >2014-10-22 16:45:32,464 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVmsInfoVDSCommand] (ajp-/127.0.0.1:8702-3) START, GetVmsInfoVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = 45be96a3-8ccc-4807-8bd6-ff4960c451aa, vmIdList = null), log id: 2f44bb4f >2014-10-22 16:45:32,500 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVmsInfoVDSCommand] (ajp-/127.0.0.1:8702-3) FINISH, GetVmsInfoVDSCommand, log id: 2f44bb4f >2014-10-22 16:45:33,136 INFO [org.ovirt.engine.core.bll.ExportVmCommand] (ajp-/127.0.0.1:8702-2) [4950247a] Lock Acquired to object EngineLock [exclusiveLocks= key: 3e13f009-03f3-4a43-b540-ad7fe3122e53 value: VM >, sharedLocks= ] >2014-10-22 16:45:33,195 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVmsInfoVDSCommand] (ajp-/127.0.0.1:8702-2) [4950247a] START, GetVmsInfoVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = 45be96a3-8ccc-4807-8bd6-ff4960c451aa, vmIdList = null), log id: 34f0e2ea >2014-10-22 16:45:33,314 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVmsInfoVDSCommand] (ajp-/127.0.0.1:8702-2) [4950247a] FINISH, GetVmsInfoVDSCommand, log id: 34f0e2ea >2014-10-22 16:45:33,871 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVmsInfoVDSCommand] (ajp-/127.0.0.1:8702-2) [4950247a] START, GetVmsInfoVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = 45be96a3-8ccc-4807-8bd6-ff4960c451aa, vmIdList = null), log id: 26c85ca6 >2014-10-22 16:45:33,902 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVmsInfoVDSCommand] (ajp-/127.0.0.1:8702-2) [4950247a] FINISH, GetVmsInfoVDSCommand, log id: 26c85ca6 >2014-10-22 16:45:34,567 INFO [org.ovirt.engine.core.bll.ExportVmCommand] (org.ovirt.thread.pool-7-thread-3) [4950247a] Running command: ExportVmCommand internal: false. Entities affected : ID: 45be96a3-8ccc-4807-8bd6-ff4960c451aa Type: StorageAction group IMPORT_EXPORT_VM with role type ADMIN >2014-10-22 16:45:34,572 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-3) [4950247a] START, SetVmStatusVDSCommand( vmId = 3e13f009-03f3-4a43-b540-ad7fe3122e53, status = ImageLocked, exit status = Normal), log id: 4abb0d7a >2014-10-22 16:45:34,590 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-3) [4950247a] FINISH, SetVmStatusVDSCommand, log id: 4abb0d7a >2014-10-22 16:45:34,593 INFO [org.ovirt.engine.core.bll.ExportVmCommand] (org.ovirt.thread.pool-7-thread-3) [4950247a] Lock freed to object EngineLock [exclusiveLocks= key: 3e13f009-03f3-4a43-b540-ad7fe3122e53 value: VM >, sharedLocks= ] >2014-10-22 16:45:34,628 INFO [org.ovirt.engine.core.bll.CopyImageGroupCommand] (org.ovirt.thread.pool-7-thread-3) [5acd3e74] Running command: CopyImageGroupCommand internal: true. Entities affected : ID: 45be96a3-8ccc-4807-8bd6-ff4960c451aa Type: Storage >2014-10-22 16:45:34,816 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.MoveImageGroupVDSCommand] (org.ovirt.thread.pool-7-thread-3) [5acd3e74] START, MoveImageGroupVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = e3162e04-764d-40f0-993c-34865796c725, dstDomainId = 45be96a3-8ccc-4807-8bd6-ff4960c451aa, vmId = 3e13f009-03f3-4a43-b540-ad7fe3122e53, op = Copy, postZero = false, force = false), log id: 1170c20a >2014-10-22 16:45:34,997 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.MoveImageGroupVDSCommand] (org.ovirt.thread.pool-7-thread-3) [5acd3e74] FINISH, MoveImageGroupVDSCommand, log id: 1170c20a >2014-10-22 16:45:35,003 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-3) [5acd3e74] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command fd6ea25f-3f5a-426a-890f-402531101c15 >2014-10-22 16:45:35,003 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-7-thread-3) [5acd3e74] CommandMultiAsyncTasks::AttachTask: Attaching task 18967025-5f50-4e7d-924b-7e25d90cb56a to command fd6ea25f-3f5a-426a-890f-402531101c15. >2014-10-22 16:45:35,022 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-7-thread-3) [5acd3e74] Adding task 18967025-5f50-4e7d-924b-7e25d90cb56a (Parent Command ExportVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-10-22 16:45:35,051 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-3) [5acd3e74] Correlation ID: 4950247a, Job ID: f4ec7c84-fb23-4bfc-8cac-a800c923a25b, Call Stack: null, Custom Event ID: -1, Message: Starting export Vm baloon_vm to export_domain >2014-10-22 16:45:35,051 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-3) [5acd3e74] BaseAsyncTask::startPollingTask: Starting to poll task 18967025-5f50-4e7d-924b-7e25d90cb56a. >2014-10-22 16:45:39,067 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-42) Polling and updating Async Tasks: 1 tasks, 1 tasks to poll now >2014-10-22 16:45:39,638 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-42) SPMAsyncTask::PollTask: Polling task 18967025-5f50-4e7d-924b-7e25d90cb56a (Parent Command ExportVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status running. >2014-10-22 16:45:39,638 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-42) Finished polling Tasks, will poll again in 10 seconds. >2014-10-22 16:46:11,229 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-54) SPMAsyncTask::PollTask: Polling task 18967025-5f50-4e7d-924b-7e25d90cb56a (Parent Command ExportVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-10-22 16:46:11,234 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-54) BaseAsyncTask::onTaskEndSuccess: Task 18967025-5f50-4e7d-924b-7e25d90cb56a (Parent Command ExportVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-10-22 16:46:11,235 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-54) CommandAsyncTask::endActionIfNecessary: All tasks of command fd6ea25f-3f5a-426a-890f-402531101c15 has ended -> executing endAction >2014-10-22 16:46:11,235 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-54) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: fd6ea25f-3f5a-426a-890f-402531101c15): calling endAction . >2014-10-22 16:46:11,235 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-42) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction ExportVm, executionIndex: 0 >2014-10-22 16:46:11,249 INFO [org.ovirt.engine.core.bll.ExportVmCommand] (org.ovirt.thread.pool-7-thread-42) Ending command successfully: org.ovirt.engine.core.bll.ExportVmCommand >2014-10-22 16:46:11,279 INFO [org.ovirt.engine.core.bll.CopyImageGroupCommand] (org.ovirt.thread.pool-7-thread-42) [5acd3e74] Ending command successfully: org.ovirt.engine.core.bll.CopyImageGroupCommand >2014-10-22 16:46:11,282 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-42) [5acd3e74] START, GetImageInfoVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = e3162e04-764d-40f0-993c-34865796c725, imageId = 4e6c4c48-d697-46ce-9fe2-3782b0516b73), log id: 607c80b4 >2014-10-22 16:46:11,334 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-42) [5acd3e74] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@626bc7e3, log id: 607c80b4 >2014-10-22 16:46:11,385 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UpdateVMVDSCommand] (org.ovirt.thread.pool-7-thread-42) START, UpdateVMVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = 45be96a3-8ccc-4807-8bd6-ff4960c451aa, infoDictionary.size = 1), log id: 56a67cf7 >2014-10-22 16:46:11,408 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UpdateVMVDSCommand] (org.ovirt.thread.pool-7-thread-42) FINISH, UpdateVMVDSCommand, log id: 56a67cf7 >2014-10-22 16:46:11,408 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-42) START, SetVmStatusVDSCommand( vmId = 3e13f009-03f3-4a43-b540-ad7fe3122e53, status = Down, exit status = Normal), log id: 5201622 >2014-10-22 16:46:11,412 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-42) FINISH, SetVmStatusVDSCommand, log id: 5201622 >2014-10-22 16:46:11,424 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-42) Correlation ID: 4950247a, Call Stack: null, Custom Event ID: -1, Message: Vm baloon_vm was exported successfully to export_domain >2014-10-22 16:46:11,424 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-42) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type ExportVm completed, handling the result. >2014-10-22 16:46:11,424 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-42) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type ExportVm succeeded, clearing tasks. >2014-10-22 16:46:11,427 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-42) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 18967025-5f50-4e7d-924b-7e25d90cb56a >2014-10-22 16:46:11,428 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-42) START, SPMClearTaskVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, taskId = 18967025-5f50-4e7d-924b-7e25d90cb56a), log id: 58a33b87 >2014-10-22 16:46:11,430 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-42) START, HSMClearTaskVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, taskId=18967025-5f50-4e7d-924b-7e25d90cb56a), log id: 2baa6140 >2014-10-22 16:46:11,459 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-42) FINISH, HSMClearTaskVDSCommand, log id: 2baa6140 >2014-10-22 16:46:11,460 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-42) FINISH, SPMClearTaskVDSCommand, log id: 58a33b87 >2014-10-22 16:46:11,465 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-42) BaseAsyncTask::removeTaskFromDB: Removed task 18967025-5f50-4e7d-924b-7e25d90cb56a from DataBase >2014-10-22 16:46:11,465 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-42) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity fd6ea25f-3f5a-426a-890f-402531101c15 >2014-10-22 16:46:31,259 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (ajp-/127.0.0.1:8702-8) [2569968f] Lock Acquired to object EngineLock [exclusiveLocks= key: 3e13f009-03f3-4a43-b540-ad7fe3122e53 value: VM >, sharedLocks= ] >2014-10-22 16:46:31,304 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-7) [2569968f] Running command: RemoveVmCommand internal: false. Entities affected : ID: 3e13f009-03f3-4a43-b540-ad7fe3122e53 Type: VMAction group DELETE_VM with role type USER >2014-10-22 16:46:31,308 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-7) [2569968f] START, SetVmStatusVDSCommand( vmId = 3e13f009-03f3-4a43-b540-ad7fe3122e53, status = ImageLocked, exit status = Normal), log id: 55b16dc8 >2014-10-22 16:46:31,323 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-7) [2569968f] FINISH, SetVmStatusVDSCommand, log id: 55b16dc8 >2014-10-22 16:46:31,336 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-7) [2569968f] Lock freed to object EngineLock [exclusiveLocks= key: 3e13f009-03f3-4a43-b540-ad7fe3122e53 value: VM >, sharedLocks= ] >2014-10-22 16:46:31,375 INFO [org.ovirt.engine.core.bll.RemoveAllVmImagesCommand] (org.ovirt.thread.pool-7-thread-7) [4b292f32] Running command: RemoveAllVmImagesCommand internal: true. Entities affected : ID: 3e13f009-03f3-4a43-b540-ad7fe3122e53 Type: VM >2014-10-22 16:46:31,407 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-7-thread-7) [29277141] Running command: RemoveImageCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage >2014-10-22 16:46:31,622 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-7-thread-7) [29277141] START, DeleteImageGroupVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = e3162e04-764d-40f0-993c-34865796c725, postZeros = false, forceDelete = false), log id: 428c42ec >2014-10-22 16:46:32,070 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-7-thread-7) [29277141] FINISH, DeleteImageGroupVDSCommand, log id: 428c42ec >2014-10-22 16:46:32,075 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-7) [29277141] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 6bd08cd0-5bff-4c76-809a-605fbf9971f8 >2014-10-22 16:46:32,075 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-7-thread-7) [29277141] CommandMultiAsyncTasks::AttachTask: Attaching task 2d532b3d-1156-4f49-8d38-44bf8c7cc3e5 to command 6bd08cd0-5bff-4c76-809a-605fbf9971f8. >2014-10-22 16:46:32,091 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-7-thread-7) [29277141] Adding task 2d532b3d-1156-4f49-8d38-44bf8c7cc3e5 (Parent Command RemoveVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-10-22 16:46:32,333 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-7) [29277141] Correlation ID: 2569968f, Job ID: a458d8aa-ec7c-40c7-a855-c36e93597474, Call Stack: null, Custom Event ID: -1, Message: VM baloon_vm was successfully removed. >2014-10-22 16:46:32,333 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-7) [29277141] BaseAsyncTask::startPollingTask: Starting to poll task 2d532b3d-1156-4f49-8d38-44bf8c7cc3e5. >2014-10-22 16:46:40,871 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVmsInfoVDSCommand] (ajp-/127.0.0.1:8702-1) START, GetVmsInfoVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = 45be96a3-8ccc-4807-8bd6-ff4960c451aa, vmIdList = null), log id: 489bc7f0 >2014-10-22 16:46:40,911 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVmsInfoVDSCommand] (ajp-/127.0.0.1:8702-1) FINISH, GetVmsInfoVDSCommand, log id: 489bc7f0 >2014-10-22 16:46:41,256 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-55) Polling and updating Async Tasks: 2 tasks, 1 tasks to poll now >2014-10-22 16:46:41,275 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-55) SPMAsyncTask::PollTask: Polling task 2d532b3d-1156-4f49-8d38-44bf8c7cc3e5 (Parent Command RemoveVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-10-22 16:46:41,301 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-55) BaseAsyncTask::onTaskEndSuccess: Task 2d532b3d-1156-4f49-8d38-44bf8c7cc3e5 (Parent Command RemoveVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-10-22 16:46:41,310 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-55) CommandAsyncTask::endActionIfNecessary: All tasks of command 6bd08cd0-5bff-4c76-809a-605fbf9971f8 has ended -> executing endAction >2014-10-22 16:46:41,310 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-55) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: 6bd08cd0-5bff-4c76-809a-605fbf9971f8): calling endAction . >2014-10-22 16:46:41,310 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-19) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction RemoveVm, executionIndex: 0 >2014-10-22 16:46:41,475 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-19) Ending command successfully: org.ovirt.engine.core.bll.RemoveVmCommand >2014-10-22 16:46:41,477 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-19) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RemoveVm completed, handling the result. >2014-10-22 16:46:41,477 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-19) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RemoveVm succeeded, clearing tasks. >2014-10-22 16:46:41,501 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-19) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 2d532b3d-1156-4f49-8d38-44bf8c7cc3e5 >2014-10-22 16:46:41,502 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-19) START, SPMClearTaskVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, taskId = 2d532b3d-1156-4f49-8d38-44bf8c7cc3e5), log id: 41a6dfac >2014-10-22 16:46:41,561 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-19) START, HSMClearTaskVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, taskId=2d532b3d-1156-4f49-8d38-44bf8c7cc3e5), log id: 32a6b6e0 >2014-10-22 16:46:41,600 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-19) FINISH, HSMClearTaskVDSCommand, log id: 32a6b6e0 >2014-10-22 16:46:41,600 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-19) FINISH, SPMClearTaskVDSCommand, log id: 41a6dfac >2014-10-22 16:46:41,614 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-19) BaseAsyncTask::removeTaskFromDB: Removed task 2d532b3d-1156-4f49-8d38-44bf8c7cc3e5 from DataBase >2014-10-22 16:46:41,614 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-19) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 6bd08cd0-5bff-4c76-809a-605fbf9971f8 >2014-10-22 16:46:52,986 INFO [org.ovirt.engine.core.bll.ImportVmCommand] (ajp-/127.0.0.1:8702-2) [46c67047] Lock Acquired to object EngineLock [exclusiveLocks= key: baloon_vm value: VM_NAME >, sharedLocks= key: 3e13f009-03f3-4a43-b540-ad7fe3122e53 value: REMOTE_VM >] >2014-10-22 16:46:52,997 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVmsInfoVDSCommand] (ajp-/127.0.0.1:8702-2) [46c67047] START, GetVmsInfoVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = 45be96a3-8ccc-4807-8bd6-ff4960c451aa, vmIdList = null), log id: 56ce939f >2014-10-22 16:46:53,027 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVmsInfoVDSCommand] (ajp-/127.0.0.1:8702-2) [46c67047] FINISH, GetVmsInfoVDSCommand, log id: 56ce939f >2014-10-22 16:46:53,683 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImagesListVDSCommand] (ajp-/127.0.0.1:8702-2) [46c67047] START, GetImagesListVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, sdUUID = a1341057-9cf8-4750-a734-656d05a5d2c5), log id: 60ca9d92 >2014-10-22 16:46:53,730 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImagesListVDSCommand] (ajp-/127.0.0.1:8702-2) [46c67047] FINISH, GetImagesListVDSCommand, return: [416239b4-ea7f-445b-a347-a4b44aa901b4, b12fe15d-81b9-4fa2-94c3-b7b2f90f1b4d, 2e7089ff-2fca-4243-b58b-dcef3678ca02, 9a68745c-3578-46d7-ad9c-d9e03c4639dd, 493ae1d9-5bb7-4948-a637-3e5bc861ac45, 65766f65-44ba-4e98-9e73-3d1fea2106e4, f988c27d-db38-4446-b082-2c4107280333, dfba228f-46f3-4758-a4f0-05857683a088, 3df49b50-c82d-42d0-b656-9877ca444e01, 3778d539-c7de-48fc-be30-9a02bf87665a, 2cc3f4d2-4b42-49b6-9260-5b604d5ffa81, 40c0302a-f7f6-49a2-97a7-3aee6b07d502, 1742b7af-bb81-4e6b-bf10-feab68a2f799], log id: 60ca9d92 >2014-10-22 16:46:53,731 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DoesImageExistVDSCommand] (ajp-/127.0.0.1:8702-2) [46c67047] START, DoesImageExistVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = 45be96a3-8ccc-4807-8bd6-ff4960c451aa, imageGroupId = e3162e04-764d-40f0-993c-34865796c725, imageId = 4e6c4c48-d697-46ce-9fe2-3782b0516b73), log id: 6786e538 >2014-10-22 16:46:53,767 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DoesImageExistVDSCommand] (ajp-/127.0.0.1:8702-2) [46c67047] FINISH, DoesImageExistVDSCommand, return: true, log id: 6786e538 >2014-10-22 16:46:53,881 INFO [org.ovirt.engine.core.bll.ImportVmCommand] (org.ovirt.thread.pool-7-thread-33) [46c67047] Running command: ImportVmCommand internal: false. Entities affected : ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: StorageAction group IMPORT_EXPORT_VM with role type ADMIN >2014-10-22 16:46:53,968 INFO [org.ovirt.engine.core.bll.CopyImageGroupCommand] (org.ovirt.thread.pool-7-thread-33) [481810ac] Running command: CopyImageGroupCommand internal: true. Entities affected : ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: Storage >2014-10-22 16:46:54,308 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (org.ovirt.thread.pool-7-thread-33) [481810ac] START, CopyImageVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = 45be96a3-8ccc-4807-8bd6-ff4960c451aa, imageGroupId = e3162e04-764d-40f0-993c-34865796c725, imageId = 4e6c4c48-d697-46ce-9fe2-3782b0516b73, dstImageGroupId = e3162e04-764d-40f0-993c-34865796c725, vmId = 3e13f009-03f3-4a43-b540-ad7fe3122e53, dstImageId = 4e6c4c48-d697-46ce-9fe2-3782b0516b73, imageDescription = , dstStorageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, copyVolumeType = LeafVol, volumeFormat = RAW, preallocate = Sparse, postZero = false, force = true), log id: 709d037d >2014-10-22 16:46:54,309 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (org.ovirt.thread.pool-7-thread-33) [481810ac] -- executeIrsBrokerCommand: calling 'copyImage' with two new parameters: description and UUID >2014-10-22 16:46:54,309 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (org.ovirt.thread.pool-7-thread-33) [481810ac] -- copyImage parameters: > sdUUID=45be96a3-8ccc-4807-8bd6-ff4960c451aa > spUUID=f8f5eaee-8fd0-4b45-87db-62d61b03a916 > vmGUID=3e13f009-03f3-4a43-b540-ad7fe3122e53 > srcImageGUID=e3162e04-764d-40f0-993c-34865796c725 > srcVolUUID=4e6c4c48-d697-46ce-9fe2-3782b0516b73 > dstImageGUID=e3162e04-764d-40f0-993c-34865796c725 > dstVolUUID=4e6c4c48-d697-46ce-9fe2-3782b0516b73 > descr= > dstSdUUID=a1341057-9cf8-4750-a734-656d05a5d2c5 >2014-10-22 16:46:54,643 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (org.ovirt.thread.pool-7-thread-33) [481810ac] FINISH, CopyImageVDSCommand, return: 00000000-0000-0000-0000-000000000000, log id: 709d037d >2014-10-22 16:46:54,648 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-33) [481810ac] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 0ecabdd0-bd5d-4428-83c4-d3b415eb1397 >2014-10-22 16:46:54,648 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-7-thread-33) [481810ac] CommandMultiAsyncTasks::AttachTask: Attaching task fe61c47a-2ecd-4a60-8e59-5b3854f5a18a to command 0ecabdd0-bd5d-4428-83c4-d3b415eb1397. >2014-10-22 16:46:54,663 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-7-thread-33) [481810ac] Adding task fe61c47a-2ecd-4a60-8e59-5b3854f5a18a (Parent Command ImportVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-10-22 16:46:54,712 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-33) [481810ac] START, SetVmStatusVDSCommand( vmId = 3e13f009-03f3-4a43-b540-ad7fe3122e53, status = ImageLocked, exit status = Normal), log id: 6ad2174e >2014-10-22 16:46:54,715 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-33) [481810ac] FINISH, SetVmStatusVDSCommand, log id: 6ad2174e >2014-10-22 16:46:54,772 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-33) [481810ac] Correlation ID: 46c67047, Job ID: ae20bc00-4279-4ab9-b891-2cd013101b75, Call Stack: null, Custom Event ID: -1, Message: Starting to import Vm baloon_vm to Data Center dc_35, Cluster cl_35 >2014-10-22 16:46:54,772 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-33) [481810ac] BaseAsyncTask::startPollingTask: Starting to poll task fe61c47a-2ecd-4a60-8e59-5b3854f5a18a. >2014-10-22 16:47:01,311 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-18) Polling and updating Async Tasks: 3 tasks, 1 tasks to poll now >2014-10-22 16:47:01,351 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-18) SPMAsyncTask::PollTask: Polling task fe61c47a-2ecd-4a60-8e59-5b3854f5a18a (Parent Command ImportVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status running. >2014-10-22 16:47:01,351 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-18) Finished polling Tasks, will poll again in 10 seconds. >2014-10-22 16:47:18,906 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-84) Setting new tasks map. The map contains now 2 tasks >2014-10-22 16:47:21,363 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-97) Polling and updating Async Tasks: 2 tasks, 1 tasks to poll now >2014-10-22 16:47:21,375 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-97) SPMAsyncTask::PollTask: Polling task fe61c47a-2ecd-4a60-8e59-5b3854f5a18a (Parent Command ImportVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-10-22 16:47:21,379 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-97) BaseAsyncTask::onTaskEndSuccess: Task fe61c47a-2ecd-4a60-8e59-5b3854f5a18a (Parent Command ImportVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-10-22 16:47:21,380 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-97) CommandAsyncTask::endActionIfNecessary: All tasks of command 0ecabdd0-bd5d-4428-83c4-d3b415eb1397 has ended -> executing endAction >2014-10-22 16:47:21,380 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-97) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: 0ecabdd0-bd5d-4428-83c4-d3b415eb1397): calling endAction . >2014-10-22 16:47:21,381 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-5) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction ImportVm, executionIndex: 0 >2014-10-22 16:47:21,422 INFO [org.ovirt.engine.core.bll.ImportVmCommand] (org.ovirt.thread.pool-7-thread-5) Ending command successfully: org.ovirt.engine.core.bll.ImportVmCommand >2014-10-22 16:47:21,430 INFO [org.ovirt.engine.core.bll.CopyImageGroupCommand] (org.ovirt.thread.pool-7-thread-5) [481810ac] Ending command successfully: org.ovirt.engine.core.bll.CopyImageGroupCommand >2014-10-22 16:47:21,445 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-5) [481810ac] START, GetImageInfoVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = e3162e04-764d-40f0-993c-34865796c725, imageId = 4e6c4c48-d697-46ce-9fe2-3782b0516b73), log id: 7ecae47f >2014-10-22 16:47:21,513 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-5) [481810ac] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@626bc7e3, log id: 7ecae47f >2014-10-22 16:47:21,521 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-5) START, SetVmStatusVDSCommand( vmId = 3e13f009-03f3-4a43-b540-ad7fe3122e53, status = Down, exit status = Normal), log id: 6ff3cf8a >2014-10-22 16:47:21,524 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-5) FINISH, SetVmStatusVDSCommand, log id: 6ff3cf8a >2014-10-22 16:47:21,525 INFO [org.ovirt.engine.core.bll.ImportVmCommand] (org.ovirt.thread.pool-7-thread-5) Lock freed to object EngineLock [exclusiveLocks= key: baloon_vm value: VM_NAME >, sharedLocks= key: 3e13f009-03f3-4a43-b540-ad7fe3122e53 value: REMOTE_VM >] >2014-10-22 16:47:21,532 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-5) Correlation ID: 46c67047, Job ID: ae20bc00-4279-4ab9-b891-2cd013101b75, Call Stack: null, Custom Event ID: -1, Message: Vm baloon_vm was imported successfully to Data Center dc_35, Cluster cl_35 >2014-10-22 16:47:21,532 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-5) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type ImportVm completed, handling the result. >2014-10-22 16:47:21,532 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-5) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type ImportVm succeeded, clearing tasks. >2014-10-22 16:47:21,535 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-5) SPMAsyncTask::ClearAsyncTask: Attempting to clear task fe61c47a-2ecd-4a60-8e59-5b3854f5a18a >2014-10-22 16:47:21,536 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-5) START, SPMClearTaskVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, taskId = fe61c47a-2ecd-4a60-8e59-5b3854f5a18a), log id: 5f24c23 >2014-10-22 16:47:21,537 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-5) START, HSMClearTaskVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, taskId=fe61c47a-2ecd-4a60-8e59-5b3854f5a18a), log id: 666490e1 >2014-10-22 16:47:21,585 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-5) FINISH, HSMClearTaskVDSCommand, log id: 666490e1 >2014-10-22 16:47:21,585 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-5) FINISH, SPMClearTaskVDSCommand, log id: 5f24c23 >2014-10-22 16:47:21,590 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-5) BaseAsyncTask::removeTaskFromDB: Removed task fe61c47a-2ecd-4a60-8e59-5b3854f5a18a from DataBase >2014-10-22 16:47:21,590 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-5) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 0ecabdd0-bd5d-4428-83c4-d3b415eb1397 >2014-10-22 16:47:48,907 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-86) Setting new tasks map. The map contains now 1 tasks >2014-10-22 16:48:09,400 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-7) Running command: LoginUserCommand internal: false. >2014-10-22 16:48:09,402 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-10-22 16:48:09,605 INFO [org.ovirt.engine.core.bll.aaa.LogoutBySessionCommand] (ajp-/127.0.0.1:8702-7) [3a9f61ac] Running command: LogoutBySessionCommand internal: false. >2014-10-22 16:48:09,608 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-7) [51c5d559] Running command: LogoutUserCommand internal: false. >2014-10-22 16:48:09,614 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [51c5d559] Correlation ID: 51c5d559, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-10-22 16:48:19,871 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-5) Running command: LoginUserCommand internal: false. >2014-10-22 16:48:19,874 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-10-22 16:48:19,914 INFO [org.ovirt.engine.core.bll.aaa.LogoutBySessionCommand] (ajp-/127.0.0.1:8702-5) [7db7e6cb] Running command: LogoutBySessionCommand internal: false. >2014-10-22 16:48:19,917 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-5) [5f106994] Running command: LogoutUserCommand internal: false. >2014-10-22 16:48:19,920 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [5f106994] Correlation ID: 5f106994, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-10-22 16:48:48,909 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-92) Setting new tasks map. The map contains now 0 tasks >2014-10-22 16:48:48,909 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-92) Cleared all tasks of pool f8f5eaee-8fd0-4b45-87db-62d61b03a916. >2014-10-22 16:49:06,537 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-8) Running command: LoginUserCommand internal: false. >2014-10-22 16:49:06,541 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-8) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-10-22 17:32:19,135 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-96) Principal admin::internal synchronized >2014-10-22 17:32:19,809 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-94) Attempting to update VMs/Templates Ovf. >2014-10-22 17:32:20,119 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-94) [33dd0e18] Lock Acquired to object EngineLock [exclusiveLocks= key: 1742b7af-bb81-4e6b-bf10-feab68a2f799 value: DISK >key: a1341057-9cf8-4750-a734-656d05a5d2c5 value: STORAGE >key: dfba228f-46f3-4758-a4f0-05857683a088 value: DISK >, sharedLocks= key: f8f5eaee-8fd0-4b45-87db-62d61b03a916 value: OVF_UPDATE >] >2014-10-22 17:32:20,123 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-94) [33dd0e18] Running command: ProcessOvfUpdateForStorageDomainCommand internal: true. Entities affected : ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: Storage >2014-10-22 17:32:20,192 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-94) [33dd0e18] START, SetVolumeDescriptionVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = dfba228f-46f3-4758-a4f0-05857683a088, imageId = 26155adf-9478-4927-8c49-8e12088c8648), log id: 2357250a >2014-10-22 17:32:20,192 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-94) [33dd0e18] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-10-22 17:32:20,192 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-94) [33dd0e18] -- setVolumeDescription parameters: > spUUID=f8f5eaee-8fd0-4b45-87db-62d61b03a916 > sdUUID=a1341057-9cf8-4750-a734-656d05a5d2c5 > imageGroupGUID=dfba228f-46f3-4758-a4f0-05857683a088 > volUUID=26155adf-9478-4927-8c49-8e12088c8648 > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"a1341057-9cf8-4750-a734-656d05a5d2c5"}],"Last Updated":"Mon Oct 20 16:04:09 IDT 2014"} > >2014-10-22 17:32:20,277 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-94) [33dd0e18] FINISH, SetVolumeDescriptionVDSCommand, log id: 2357250a >2014-10-22 17:32:20,430 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-94) [36b3e379] Running command: UploadStreamCommand internal: true. Entities affected : ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: Storage >2014-10-22 17:32:20,442 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-94) [36b3e379] START, UploadStreamVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6), log id: 19a538c4 >2014-10-22 17:32:20,442 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-94) [36b3e379] -- executeVdsBrokerCommand: >2014-10-22 17:32:20,442 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-94) [36b3e379] -- parameters: > spUUID=f8f5eaee-8fd0-4b45-87db-62d61b03a916 > sdUUID=a1341057-9cf8-4750-a734-656d05a5d2c5 > imageGUID=dfba228f-46f3-4758-a4f0-05857683a088 > volUUID=26155adf-9478-4927-8c49-8e12088c8648 > size=71,680 > >2014-10-22 17:32:22,613 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-94) [36b3e379] FINISH, UploadStreamVDSCommand, log id: 19a538c4 >2014-10-22 17:32:22,617 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-94) [36b3e379] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 4ada0779-5ce6-411a-8ace-dbc5c8d4d842 >2014-10-22 17:32:22,617 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-94) [36b3e379] CommandMultiAsyncTasks::AttachTask: Attaching task e768f960-9a62-44fa-959b-ddcef8c56b61 to command 4ada0779-5ce6-411a-8ace-dbc5c8d4d842. >2014-10-22 17:32:22,627 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-94) [36b3e379] Adding task e768f960-9a62-44fa-959b-ddcef8c56b61 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-10-22 17:32:22,631 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-94) [36b3e379] Lock freed to object EngineLock [exclusiveLocks= key: 1742b7af-bb81-4e6b-bf10-feab68a2f799 value: DISK >key: a1341057-9cf8-4750-a734-656d05a5d2c5 value: STORAGE >key: dfba228f-46f3-4758-a4f0-05857683a088 value: DISK >, sharedLocks= key: f8f5eaee-8fd0-4b45-87db-62d61b03a916 value: OVF_UPDATE >] >2014-10-22 17:32:22,636 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-94) [36b3e379] START, SetVolumeDescriptionVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = dfba228f-46f3-4758-a4f0-05857683a088, imageId = 26155adf-9478-4927-8c49-8e12088c8648), log id: 4797e2e0 >2014-10-22 17:32:22,636 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-94) [36b3e379] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-10-22 17:32:22,637 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-94) [36b3e379] -- setVolumeDescription parameters: > spUUID=f8f5eaee-8fd0-4b45-87db-62d61b03a916 > sdUUID=a1341057-9cf8-4750-a734-656d05a5d2c5 > imageGroupGUID=dfba228f-46f3-4758-a4f0-05857683a088 > volUUID=26155adf-9478-4927-8c49-8e12088c8648 > description={"Updated":true,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"a1341057-9cf8-4750-a734-656d05a5d2c5"}],"Last Updated":"Wed Oct 22 17:32:20 IDT 2014","Size":71680} > >2014-10-22 17:32:22,730 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-94) [36b3e379] FINISH, SetVolumeDescriptionVDSCommand, log id: 4797e2e0 >2014-10-22 17:32:22,737 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-94) [36b3e379] START, SetVolumeDescriptionVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = 1742b7af-bb81-4e6b-bf10-feab68a2f799, imageId = 5b2b66c8-425d-4c49-9b3c-62f7586aac1b), log id: 26c3cf22 >2014-10-22 17:32:22,737 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-94) [36b3e379] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-10-22 17:32:22,737 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-94) [36b3e379] -- setVolumeDescription parameters: > spUUID=f8f5eaee-8fd0-4b45-87db-62d61b03a916 > sdUUID=a1341057-9cf8-4750-a734-656d05a5d2c5 > imageGroupGUID=1742b7af-bb81-4e6b-bf10-feab68a2f799 > volUUID=5b2b66c8-425d-4c49-9b3c-62f7586aac1b > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"a1341057-9cf8-4750-a734-656d05a5d2c5"}],"Last Updated":"Mon Oct 20 16:04:09 IDT 2014"} > >2014-10-22 17:32:22,815 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-94) [36b3e379] FINISH, SetVolumeDescriptionVDSCommand, log id: 26c3cf22 >2014-10-22 17:32:22,837 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-94) [2b987d4e] Running command: UploadStreamCommand internal: true. Entities affected : ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: Storage >2014-10-22 17:32:22,839 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-94) [2b987d4e] START, UploadStreamVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6), log id: 7cd9f1cc >2014-10-22 17:32:22,839 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-94) [2b987d4e] -- executeVdsBrokerCommand: >2014-10-22 17:32:22,839 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-94) [2b987d4e] -- parameters: > spUUID=f8f5eaee-8fd0-4b45-87db-62d61b03a916 > sdUUID=a1341057-9cf8-4750-a734-656d05a5d2c5 > imageGUID=1742b7af-bb81-4e6b-bf10-feab68a2f799 > volUUID=5b2b66c8-425d-4c49-9b3c-62f7586aac1b > size=71,680 > >2014-10-22 17:32:24,121 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-94) [2b987d4e] FINISH, UploadStreamVDSCommand, log id: 7cd9f1cc >2014-10-22 17:32:24,123 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-94) [2b987d4e] CommandMultiAsyncTasks::AttachTask: Attaching task d2b050ff-5cb1-4438-8899-bf33eaa6656e to command 4ada0779-5ce6-411a-8ace-dbc5c8d4d842. >2014-10-22 17:32:24,132 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-94) [2b987d4e] Adding task d2b050ff-5cb1-4438-8899-bf33eaa6656e (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-10-22 17:32:24,136 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-94) [2b987d4e] Trying to release a shared lock for key: f8f5eaee-8fd0-4b45-87db-62d61b03a916OVF_UPDATE , but lock does not exist >2014-10-22 17:32:24,136 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-94) [2b987d4e] Trying to release exclusive lock which does not exist, lock key: 1742b7af-bb81-4e6b-bf10-feab68a2f799DISK >2014-10-22 17:32:24,136 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-94) [2b987d4e] Trying to release exclusive lock which does not exist, lock key: a1341057-9cf8-4750-a734-656d05a5d2c5STORAGE >2014-10-22 17:32:24,137 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-94) [2b987d4e] Trying to release exclusive lock which does not exist, lock key: dfba228f-46f3-4758-a4f0-05857683a088DISK >2014-10-22 17:32:24,137 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-94) [2b987d4e] Lock freed to object EngineLock [exclusiveLocks= key: 1742b7af-bb81-4e6b-bf10-feab68a2f799 value: DISK >key: a1341057-9cf8-4750-a734-656d05a5d2c5 value: STORAGE >key: dfba228f-46f3-4758-a4f0-05857683a088 value: DISK >, sharedLocks= key: f8f5eaee-8fd0-4b45-87db-62d61b03a916 value: OVF_UPDATE >] >2014-10-22 17:32:24,138 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-94) [2b987d4e] START, SetVolumeDescriptionVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = 1742b7af-bb81-4e6b-bf10-feab68a2f799, imageId = 5b2b66c8-425d-4c49-9b3c-62f7586aac1b), log id: efc732e >2014-10-22 17:32:24,204 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-94) [2b987d4e] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-10-22 17:32:24,204 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-94) [2b987d4e] -- setVolumeDescription parameters: > spUUID=f8f5eaee-8fd0-4b45-87db-62d61b03a916 > sdUUID=a1341057-9cf8-4750-a734-656d05a5d2c5 > imageGroupGUID=1742b7af-bb81-4e6b-bf10-feab68a2f799 > volUUID=5b2b66c8-425d-4c49-9b3c-62f7586aac1b > description={"Updated":true,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"a1341057-9cf8-4750-a734-656d05a5d2c5"}],"Last Updated":"Wed Oct 22 17:32:20 IDT 2014","Size":71680} > >2014-10-22 17:32:24,476 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-94) [2b987d4e] FINISH, SetVolumeDescriptionVDSCommand, log id: efc732e >2014-10-22 17:32:24,483 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-94) [2b987d4e] BaseAsyncTask::startPollingTask: Starting to poll task e768f960-9a62-44fa-959b-ddcef8c56b61. >2014-10-22 17:32:24,483 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-94) [2b987d4e] BaseAsyncTask::startPollingTask: Starting to poll task d2b050ff-5cb1-4438-8899-bf33eaa6656e. >2014-10-22 17:32:31,514 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-54) Polling and updating Async Tasks: 2 tasks, 2 tasks to poll now >2014-10-22 17:32:31,527 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-54) SPMAsyncTask::PollTask: Polling task d2b050ff-5cb1-4438-8899-bf33eaa6656e (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-10-22 17:32:31,527 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-54) BaseAsyncTask::onTaskEndSuccess: Task d2b050ff-5cb1-4438-8899-bf33eaa6656e (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-10-22 17:32:31,528 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-54) Task with DB Task ID f1f0f393-b37a-411a-9cc0-6d07d14b3a9b and VDSM Task ID e768f960-9a62-44fa-959b-ddcef8c56b61 is in state Polling. End action for command 4ada0779-5ce6-411a-8ace-dbc5c8d4d842 will proceed when all the entitys tasks are completed. >2014-10-22 17:32:31,528 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-54) SPMAsyncTask::PollTask: Polling task e768f960-9a62-44fa-959b-ddcef8c56b61 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-10-22 17:32:31,528 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-54) BaseAsyncTask::onTaskEndSuccess: Task e768f960-9a62-44fa-959b-ddcef8c56b61 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-10-22 17:32:31,528 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-54) CommandAsyncTask::endActionIfNecessary: All tasks of command 4ada0779-5ce6-411a-8ace-dbc5c8d4d842 has ended -> executing endAction >2014-10-22 17:32:31,529 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-54) CommandAsyncTask::endAction: Ending action for 2 tasks (command ID: 4ada0779-5ce6-411a-8ace-dbc5c8d4d842): calling endAction . >2014-10-22 17:32:31,529 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-8) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction ProcessOvfUpdateForStorageDomain, executionIndex: 0 >2014-10-22 17:32:31,571 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (org.ovirt.thread.pool-7-thread-8) [33dd0e18] Ending command successfully: org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand >2014-10-22 17:32:31,572 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-8) [33dd0e18] Trying to release a shared lock for key: f8f5eaee-8fd0-4b45-87db-62d61b03a916OVF_UPDATE , but lock does not exist >2014-10-22 17:32:31,578 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-8) [33dd0e18] Trying to release exclusive lock which does not exist, lock key: 1742b7af-bb81-4e6b-bf10-feab68a2f799DISK >2014-10-22 17:32:31,578 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-8) [33dd0e18] Trying to release exclusive lock which does not exist, lock key: a1341057-9cf8-4750-a734-656d05a5d2c5STORAGE >2014-10-22 17:32:31,578 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-8) [33dd0e18] Trying to release exclusive lock which does not exist, lock key: dfba228f-46f3-4758-a4f0-05857683a088DISK >2014-10-22 17:32:31,578 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (org.ovirt.thread.pool-7-thread-8) [33dd0e18] Lock freed to object EngineLock [exclusiveLocks= key: 1742b7af-bb81-4e6b-bf10-feab68a2f799 value: DISK >key: a1341057-9cf8-4750-a734-656d05a5d2c5 value: STORAGE >key: dfba228f-46f3-4758-a4f0-05857683a088 value: DISK >, sharedLocks= key: f8f5eaee-8fd0-4b45-87db-62d61b03a916 value: OVF_UPDATE >] >2014-10-22 17:32:31,578 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-8) [33dd0e18] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type ProcessOvfUpdateForStorageDomain completed, handling the result. >2014-10-22 17:32:31,578 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-8) [33dd0e18] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type ProcessOvfUpdateForStorageDomain succeeded, clearing tasks. >2014-10-22 17:32:31,578 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-8) [33dd0e18] SPMAsyncTask::ClearAsyncTask: Attempting to clear task d2b050ff-5cb1-4438-8899-bf33eaa6656e >2014-10-22 17:32:31,579 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-8) [33dd0e18] START, SPMClearTaskVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, taskId = d2b050ff-5cb1-4438-8899-bf33eaa6656e), log id: 2669ff8e >2014-10-22 17:32:31,591 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-8) [33dd0e18] START, HSMClearTaskVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, taskId=d2b050ff-5cb1-4438-8899-bf33eaa6656e), log id: 7908d755 >2014-10-22 17:32:32,136 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-8) [33dd0e18] FINISH, HSMClearTaskVDSCommand, log id: 7908d755 >2014-10-22 17:32:32,136 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-8) [33dd0e18] FINISH, SPMClearTaskVDSCommand, log id: 2669ff8e >2014-10-22 17:32:32,141 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-8) [33dd0e18] BaseAsyncTask::removeTaskFromDB: Removed task d2b050ff-5cb1-4438-8899-bf33eaa6656e from DataBase >2014-10-22 17:32:32,142 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-8) [33dd0e18] SPMAsyncTask::ClearAsyncTask: Attempting to clear task e768f960-9a62-44fa-959b-ddcef8c56b61 >2014-10-22 17:32:32,142 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-8) [33dd0e18] START, SPMClearTaskVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, taskId = e768f960-9a62-44fa-959b-ddcef8c56b61), log id: 7fcac2a >2014-10-22 17:32:32,143 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-8) [33dd0e18] START, HSMClearTaskVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, taskId=e768f960-9a62-44fa-959b-ddcef8c56b61), log id: 5d8afd7e >2014-10-22 17:32:32,194 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-8) [33dd0e18] FINISH, HSMClearTaskVDSCommand, log id: 5d8afd7e >2014-10-22 17:32:32,194 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-8) [33dd0e18] FINISH, SPMClearTaskVDSCommand, log id: 7fcac2a >2014-10-22 17:32:32,198 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-8) [33dd0e18] BaseAsyncTask::removeTaskFromDB: Removed task e768f960-9a62-44fa-959b-ddcef8c56b61 from DataBase >2014-10-22 17:32:32,198 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-8) [33dd0e18] CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 4ada0779-5ce6-411a-8ace-dbc5c8d4d842 >2014-10-22 17:32:33,112 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-47) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 17:32:36,377 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-62) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 17:33:48,949 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-15) Setting new tasks map. The map contains now 0 tasks >2014-10-22 17:33:48,949 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-15) Cleared all tasks of pool f8f5eaee-8fd0-4b45-87db-62d61b03a916. >2014-10-22 18:32:19,142 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-26) Principal admin::internal synchronized >2014-10-22 18:32:24,484 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-48) Attempting to update VMs/Templates Ovf. >2014-10-22 18:32:33,838 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-86) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 18:32:43,174 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-12) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 18:35:05,559 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-6) Running command: LoginUserCommand internal: false. >2014-10-22 18:35:05,563 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-10-22 18:35:06,407 INFO [org.ovirt.engine.core.bll.aaa.LogoutBySessionCommand] (ajp-/127.0.0.1:8702-6) [7d1bbfd8] Running command: LogoutBySessionCommand internal: false. >2014-10-22 18:35:06,411 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-6) [48ee69f1] Running command: LogoutUserCommand internal: false. >2014-10-22 18:35:06,414 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) [48ee69f1] Correlation ID: 48ee69f1, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-10-22 18:35:07,263 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-7) Running command: LoginUserCommand internal: false. >2014-10-22 19:32:19,152 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-32) Principal admin::internal synchronized >2014-10-22 19:32:24,527 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-81) Attempting to update VMs/Templates Ovf. >2014-10-22 19:32:34,792 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 19:32:49,543 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-56) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 20:32:19,164 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-52) Principal admin::internal synchronized >2014-10-22 20:32:24,561 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-88) Attempting to update VMs/Templates Ovf. >2014-10-22 20:32:35,297 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-22) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 20:32:57,371 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 21:32:19,169 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-97) Principal admin::internal synchronized >2014-10-22 21:32:24,597 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-20) Attempting to update VMs/Templates Ovf. >2014-10-22 21:32:50,454 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-3) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 21:33:03,957 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-39) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 22:32:19,175 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-85) Principal admin::internal synchronized >2014-10-22 22:32:24,617 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-29) [3a8c61ea] Attempting to update VMs/Templates Ovf. >2014-10-22 22:33:05,272 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-97) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 22:33:14,167 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-8) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 23:32:19,184 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-94) [2b987d4e] Principal admin::internal synchronized >2014-10-22 23:32:24,652 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-68) Attempting to update VMs/Templates Ovf. >2014-10-22 23:33:05,771 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-82) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-22 23:33:24,112 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-66) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-23 00:32:19,196 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-39) Principal admin::internal synchronized >2014-10-23 00:32:24,670 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-58) Attempting to update VMs/Templates Ovf. >2014-10-23 00:33:20,910 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-46) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-23 00:33:34,065 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-9) [1d7a0a03] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-23 01:32:19,203 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-51) Principal admin::internal synchronized >2014-10-23 01:32:24,689 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-82) Attempting to update VMs/Templates Ovf. >2014-10-23 01:33:21,004 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-87) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-23 01:33:41,022 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-50) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 418 seconds while maximum configured value is 300 seconds. >2014-10-23 02:32:19,209 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-77) Principal admin::internal synchronized >2014-10-23 02:32:24,707 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-17) Attempting to update VMs/Templates Ovf. >2014-10-23 02:33:36,221 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-72) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-23 02:33:50,590 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-12) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-23 03:32:19,215 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-26) Principal admin::internal synchronized >2014-10-23 03:32:24,725 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-39) Attempting to update VMs/Templates Ovf. >2014-10-23 03:33:50,923 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-38) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-23 03:33:57,681 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-97) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-23 03:35:35,001 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-75) Start deleteAgedOutAuditLogs >2014-10-23 03:35:35,009 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-31) Start deleteAgedOutCommandEntities >2014-10-23 03:35:35,020 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-31) Finished deleteAgedOutCommandEntities >2014-10-23 03:35:35,021 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-75) Finished deleteAgedOutAuditLogs >2014-10-23 04:32:19,223 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-54) Principal admin::internal synchronized >2014-10-23 04:32:24,739 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-72) Attempting to update VMs/Templates Ovf. >2014-10-23 04:34:05,795 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-25) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-23 04:34:07,905 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-34) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-23 05:32:19,230 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-37) Principal admin::internal synchronized >2014-10-23 05:32:24,754 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-82) Attempting to update VMs/Templates Ovf. >2014-10-23 05:34:17,728 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-70) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-23 05:34:20,303 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-51) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-23 06:32:19,236 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-46) Principal admin::internal synchronized >2014-10-23 06:32:24,772 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-5) Attempting to update VMs/Templates Ovf. >2014-10-23 06:34:20,406 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-10) [6a4c8f66] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-23 06:34:24,399 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-54) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-23 07:32:19,243 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-3) Principal admin::internal synchronized >2014-10-23 07:32:24,789 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-49) Attempting to update VMs/Templates Ovf. >2014-10-23 07:34:34,568 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-53) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-23 07:34:35,065 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-23 08:32:19,249 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-38) Principal admin::internal synchronized >2014-10-23 08:32:24,805 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-97) Attempting to update VMs/Templates Ovf. >2014-10-23 08:34:35,173 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-98) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-23 08:34:44,605 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-29) [3a8c61ea] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-23 09:32:19,255 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-56) Principal admin::internal synchronized >2014-10-23 09:32:24,821 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-79) Attempting to update VMs/Templates Ovf. >2014-10-23 09:34:49,722 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-69) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-23 09:34:54,672 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-87) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-23 10:32:19,261 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-86) Principal admin::internal synchronized >2014-10-23 10:32:24,838 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-82) Attempting to update VMs/Templates Ovf. >2014-10-23 10:35:03,369 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-49) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-23 10:35:04,733 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-33) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-23 11:32:19,266 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-90) Principal admin::internal synchronized >2014-10-23 11:32:24,854 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-4) Attempting to update VMs/Templates Ovf. >2014-10-23 11:35:07,001 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-68) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-23 11:35:11,580 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-23 12:32:19,270 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-7) Principal admin::internal synchronized >2014-10-23 12:32:24,882 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-10) [6a4c8f66] Attempting to update VMs/Templates Ovf. >2014-10-23 12:35:21,585 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-99) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-23 12:35:22,135 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-9) [1d7a0a03] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-23 13:13:28,732 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-9) [502ad4ab] Running command: LogoutUserCommand internal: false. >2014-10-23 13:13:28,732 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-8) [2bf04fab] Running command: LogoutUserCommand internal: false. >2014-10-23 13:13:28,736 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-6) [27f7aaf6] Running command: LogoutUserCommand internal: false. >2014-10-23 13:13:28,736 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-3) [74f910f9] Running command: LogoutUserCommand internal: false. >2014-10-23 13:13:28,742 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-7) [2137ea99] Running command: LogoutUserCommand internal: false. >2014-10-23 13:13:28,751 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) [27f7aaf6] Correlation ID: 27f7aaf6, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-10-23 13:13:28,773 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [502ad4ab] Correlation ID: 502ad4ab, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-10-23 13:13:28,777 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-2) [167ea5cf] Running command: LogoutUserCommand internal: false. >2014-10-23 13:13:28,779 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-8) [2bf04fab] Correlation ID: 2bf04fab, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-10-23 13:13:28,780 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) [74f910f9] Correlation ID: 74f910f9, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-10-23 13:13:28,782 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [2137ea99] Correlation ID: 2137ea99, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-10-23 13:13:28,791 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) [167ea5cf] Correlation ID: 167ea5cf, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-10-23 13:13:28,812 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-1) Invalid locale found in configuration: >2014-10-23 13:13:28,812 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-1) Invalid locale found in configuration: >2014-10-23 13:13:28,827 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-4) [3e78bdb2] Running command: LogoutUserCommand internal: false. >2014-10-23 13:13:28,848 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [3e78bdb2] Correlation ID: 3e78bdb2, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-10-23 13:25:04,022 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (ajp-/127.0.0.1:8702-7) Running command: LoginAdminUserCommand internal: false. >2014-10-23 13:25:04,030 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-10-23 13:25:04,079 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-1) Running command: LoginUserCommand internal: false. >2014-10-23 13:32:19,277 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-18) Principal admin::internal synchronized >2014-10-23 13:32:24,899 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-33) Attempting to update VMs/Templates Ovf. >2014-10-23 13:32:28,133 INFO [org.ovirt.engine.core.bll.AddVmPoolWithVmsCommand] (org.ovirt.thread.pool-7-thread-5) [7c60c385] Running command: AddVmPoolWithVmsCommand internal: false. Entities affected : ID: 67866b36-fd68-4106-8758-34cf31b0c3d4 Type: VdsGroupsAction group CREATE_VM_POOL with role type USER, ID: 30bad43f-c797-4445-b598-a26dbb2a3400 Type: VmTemplateAction group CREATE_VM with role type USER >2014-10-23 13:32:28,246 INFO [org.ovirt.engine.core.bll.AddVmAndAttachToPoolCommand] (org.ovirt.thread.pool-7-thread-5) [52368ad1] Running command: AddVmAndAttachToPoolCommand internal: true. Entities affected : ID: 67866b36-fd68-4106-8758-34cf31b0c3d4 Type: VdsGroups, ID: 30bad43f-c797-4445-b598-a26dbb2a3400 Type: VmTemplate, ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: StorageAction group CREATE_DISK with role type USER >2014-10-23 13:32:28,271 INFO [org.ovirt.engine.core.bll.AddVmCommand] (org.ovirt.thread.pool-7-thread-5) [344eb435] Lock Acquired to object EngineLock [exclusiveLocks= key: test_pool-1 value: VM_NAME >, sharedLocks= key: 9a68745c-3578-46d7-ad9c-d9e03c4639dd value: DISK >key: 30bad43f-c797-4445-b598-a26dbb2a3400 value: TEMPLATE >] >2014-10-23 13:32:28,354 INFO [org.ovirt.engine.core.bll.AddVmCommand] (org.ovirt.thread.pool-7-thread-5) [344eb435] Running command: AddVmCommand internal: true. Entities affected : ID: 67866b36-fd68-4106-8758-34cf31b0c3d4 Type: VdsGroupsAction group CREATE_VM with role type USER, ID: 30bad43f-c797-4445-b598-a26dbb2a3400 Type: VmTemplateAction group CREATE_VM with role type USER, ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: StorageAction group CREATE_DISK with role type USER >2014-10-23 13:32:28,448 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-5) [344eb435] START, SetVmStatusVDSCommand( vmId = 2ade22ff-8148-4126-b611-957e920ca7c6, status = ImageLocked, exit status = Normal), log id: 2961af2d >2014-10-23 13:32:28,455 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-5) [344eb435] FINISH, SetVmStatusVDSCommand, log id: 2961af2d >2014-10-23 13:32:28,561 INFO [org.ovirt.engine.core.bll.CreateSnapshotFromTemplateCommand] (org.ovirt.thread.pool-7-thread-5) [6778f2a5] Running command: CreateSnapshotFromTemplateCommand internal: true. Entities affected : ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: Storage >2014-10-23 13:32:28,566 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-7-thread-5) [6778f2a5] START, CreateSnapshotVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = 4c624442-cb01-4403-a4e0-d581d544a584, imageSizeInBytes = 1073741824, volumeFormat = COW, newImageId = 19384420-f743-488a-a94e-0207b6170453, newImageDescription = , imageId = 4ad0aced-a99f-4f8c-9ac0-a8b16ed9cf95, sourceImageGroupId = 9a68745c-3578-46d7-ad9c-d9e03c4639dd), log id: 4d5807ec >2014-10-23 13:32:28,567 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-7-thread-5) [6778f2a5] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-10-23 13:32:28,817 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-7-thread-5) [6778f2a5] FINISH, CreateSnapshotVDSCommand, return: 19384420-f743-488a-a94e-0207b6170453, log id: 4d5807ec >2014-10-23 13:32:28,820 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-5) [6778f2a5] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 44c122f4-458c-4788-8b9b-e9d5b5afb008 >2014-10-23 13:32:28,820 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-7-thread-5) [6778f2a5] CommandMultiAsyncTasks::AttachTask: Attaching task 49f15a0a-7877-415b-b236-197923e09c37 to command 44c122f4-458c-4788-8b9b-e9d5b5afb008. >2014-10-23 13:32:28,831 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-7-thread-5) [6778f2a5] Adding task 49f15a0a-7877-415b-b236-197923e09c37 (Parent Command AddVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-10-23 13:32:28,944 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-5) [6778f2a5] Correlation ID: 344eb435, Call Stack: null, Custom Event ID: -1, Message: VM test_pool-1 creation was initiated by admin. >2014-10-23 13:32:28,944 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-5) [6778f2a5] BaseAsyncTask::startPollingTask: Starting to poll task 49f15a0a-7877-415b-b236-197923e09c37. >2014-10-23 13:32:29,032 INFO [org.ovirt.engine.core.bll.AddVmToPoolCommand] (org.ovirt.thread.pool-7-thread-5) [74cf943e] Running command: AddVmToPoolCommand internal: true. Entities affected : ID: 71384dea-9dfe-4be4-b962-72e4ad0d64d0 Type: VmPool >2014-10-23 13:32:29,095 INFO [org.ovirt.engine.core.bll.AddVmAndAttachToPoolCommand] (org.ovirt.thread.pool-7-thread-5) [5d70ed9f] Running command: AddVmAndAttachToPoolCommand internal: true. Entities affected : ID: 67866b36-fd68-4106-8758-34cf31b0c3d4 Type: VdsGroups, ID: 30bad43f-c797-4445-b598-a26dbb2a3400 Type: VmTemplate, ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: StorageAction group CREATE_DISK with role type USER >2014-10-23 13:32:29,107 INFO [org.ovirt.engine.core.bll.AddVmCommand] (org.ovirt.thread.pool-7-thread-5) [13519c43] Lock Acquired to object EngineLock [exclusiveLocks= key: test_pool-2 value: VM_NAME >, sharedLocks= key: 9a68745c-3578-46d7-ad9c-d9e03c4639dd value: DISK >key: 30bad43f-c797-4445-b598-a26dbb2a3400 value: TEMPLATE >] >2014-10-23 13:32:29,148 INFO [org.ovirt.engine.core.bll.AddVmCommand] (org.ovirt.thread.pool-7-thread-5) [13519c43] Running command: AddVmCommand internal: true. Entities affected : ID: 67866b36-fd68-4106-8758-34cf31b0c3d4 Type: VdsGroupsAction group CREATE_VM with role type USER, ID: 30bad43f-c797-4445-b598-a26dbb2a3400 Type: VmTemplateAction group CREATE_VM with role type USER, ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: StorageAction group CREATE_DISK with role type USER >2014-10-23 13:32:29,203 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-5) [13519c43] START, SetVmStatusVDSCommand( vmId = 71f61d0d-8bec-41d7-86d8-e1db5ea3ab86, status = ImageLocked, exit status = Normal), log id: 46bcb880 >2014-10-23 13:32:29,257 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-5) [13519c43] FINISH, SetVmStatusVDSCommand, log id: 46bcb880 >2014-10-23 13:32:29,344 INFO [org.ovirt.engine.core.bll.CreateSnapshotFromTemplateCommand] (org.ovirt.thread.pool-7-thread-5) [4382bba0] Running command: CreateSnapshotFromTemplateCommand internal: true. Entities affected : ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: Storage >2014-10-23 13:32:29,361 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-7-thread-5) [4382bba0] START, CreateSnapshotVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = ddf27c13-9931-4730-ba37-33f1325c47c0, imageSizeInBytes = 1073741824, volumeFormat = COW, newImageId = 52814bcd-32ee-4223-bca4-5516639415da, newImageDescription = , imageId = 4ad0aced-a99f-4f8c-9ac0-a8b16ed9cf95, sourceImageGroupId = 9a68745c-3578-46d7-ad9c-d9e03c4639dd), log id: 412d2828 >2014-10-23 13:32:29,362 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-7-thread-5) [4382bba0] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-10-23 13:32:29,674 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-7-thread-5) [4382bba0] FINISH, CreateSnapshotVDSCommand, return: 52814bcd-32ee-4223-bca4-5516639415da, log id: 412d2828 >2014-10-23 13:32:29,685 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-5) [4382bba0] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 7794c7ea-85b1-4fa4-a0d7-28bd6cad32c5 >2014-10-23 13:32:29,685 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-7-thread-5) [4382bba0] CommandMultiAsyncTasks::AttachTask: Attaching task 7703a2c1-1db8-48b7-8bbf-dafa29596fa4 to command 7794c7ea-85b1-4fa4-a0d7-28bd6cad32c5. >2014-10-23 13:32:29,746 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-7-thread-5) [4382bba0] Adding task 7703a2c1-1db8-48b7-8bbf-dafa29596fa4 (Parent Command AddVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-10-23 13:32:29,856 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-5) [4382bba0] Correlation ID: 13519c43, Call Stack: null, Custom Event ID: -1, Message: VM test_pool-2 creation was initiated by admin. >2014-10-23 13:32:29,857 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-5) [4382bba0] BaseAsyncTask::startPollingTask: Starting to poll task 7703a2c1-1db8-48b7-8bbf-dafa29596fa4. >2014-10-23 13:32:29,872 INFO [org.ovirt.engine.core.bll.AddVmToPoolCommand] (org.ovirt.thread.pool-7-thread-5) [60a115cc] Running command: AddVmToPoolCommand internal: true. Entities affected : ID: 71384dea-9dfe-4be4-b962-72e4ad0d64d0 Type: VmPool >2014-10-23 13:32:29,891 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-5) [60a115cc] Correlation ID: 7c60c385, Job ID: eacdbe25-515f-4457-a057-213079d78746, Call Stack: null, Custom Event ID: -1, Message: VM Pool test_pool (containing 2 VMs) was created by admin. >2014-10-23 13:32:34,970 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-30) Polling and updating Async Tasks: 2 tasks, 2 tasks to poll now >2014-10-23 13:32:35,053 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-30) SPMAsyncTask::PollTask: Polling task 49f15a0a-7877-415b-b236-197923e09c37 (Parent Command AddVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-10-23 13:32:35,053 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-30) BaseAsyncTask::onTaskEndSuccess: Task 49f15a0a-7877-415b-b236-197923e09c37 (Parent Command AddVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-10-23 13:32:35,054 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-30) CommandAsyncTask::endActionIfNecessary: All tasks of command 44c122f4-458c-4788-8b9b-e9d5b5afb008 has ended -> executing endAction >2014-10-23 13:32:35,054 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-30) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: 44c122f4-458c-4788-8b9b-e9d5b5afb008): calling endAction . >2014-10-23 13:32:35,054 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-30) SPMAsyncTask::PollTask: Polling task 7703a2c1-1db8-48b7-8bbf-dafa29596fa4 (Parent Command AddVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-10-23 13:32:35,054 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-30) BaseAsyncTask::onTaskEndSuccess: Task 7703a2c1-1db8-48b7-8bbf-dafa29596fa4 (Parent Command AddVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-10-23 13:32:35,055 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-4) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction AddVm, executionIndex: 0 >2014-10-23 13:32:35,055 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-30) CommandAsyncTask::endActionIfNecessary: All tasks of command 7794c7ea-85b1-4fa4-a0d7-28bd6cad32c5 has ended -> executing endAction >2014-10-23 13:32:35,055 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-30) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: 7794c7ea-85b1-4fa4-a0d7-28bd6cad32c5): calling endAction . >2014-10-23 13:32:35,056 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-1) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction AddVm, executionIndex: 0 >2014-10-23 13:32:35,083 INFO [org.ovirt.engine.core.bll.AddVmCommand] (org.ovirt.thread.pool-7-thread-1) [13519c43] Ending command successfully: org.ovirt.engine.core.bll.AddVmCommand >2014-10-23 13:32:35,090 INFO [org.ovirt.engine.core.bll.AddVmCommand] (org.ovirt.thread.pool-7-thread-4) [344eb435] Ending command successfully: org.ovirt.engine.core.bll.AddVmCommand >2014-10-23 13:32:35,103 INFO [org.ovirt.engine.core.bll.CreateSnapshotFromTemplateCommand] (org.ovirt.thread.pool-7-thread-4) [6778f2a5] Ending command successfully: org.ovirt.engine.core.bll.CreateSnapshotFromTemplateCommand >2014-10-23 13:32:35,107 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-4) [6778f2a5] START, GetImageInfoVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = 4c624442-cb01-4403-a4e0-d581d544a584, imageId = 19384420-f743-488a-a94e-0207b6170453), log id: a66945c >2014-10-23 13:32:35,118 INFO [org.ovirt.engine.core.bll.CreateSnapshotFromTemplateCommand] (org.ovirt.thread.pool-7-thread-1) [4382bba0] Ending command successfully: org.ovirt.engine.core.bll.CreateSnapshotFromTemplateCommand >2014-10-23 13:32:35,132 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-1) [4382bba0] START, GetImageInfoVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = ddf27c13-9931-4730-ba37-33f1325c47c0, imageId = 52814bcd-32ee-4223-bca4-5516639415da), log id: 1259f74b >2014-10-23 13:32:35,156 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-4) [6778f2a5] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@c75ccab4, log id: a66945c >2014-10-23 13:32:35,167 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-4) START, SetVmStatusVDSCommand( vmId = 2ade22ff-8148-4126-b611-957e920ca7c6, status = Down, exit status = Normal), log id: 1cdc3641 >2014-10-23 13:32:35,172 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-4) FINISH, SetVmStatusVDSCommand, log id: 1cdc3641 >2014-10-23 13:32:35,173 INFO [org.ovirt.engine.core.bll.AddVmCommand] (org.ovirt.thread.pool-7-thread-4) Lock freed to object EngineLock [exclusiveLocks= key: test_pool-1 value: VM_NAME >, sharedLocks= key: 9a68745c-3578-46d7-ad9c-d9e03c4639dd value: DISK >key: 30bad43f-c797-4445-b598-a26dbb2a3400 value: TEMPLATE >] >2014-10-23 13:32:35,195 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-1) [4382bba0] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@f1a8ab82, log id: 1259f74b >2014-10-23 13:32:35,205 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-4) Correlation ID: 344eb435, Call Stack: null, Custom Event ID: -1, Message: VM test_pool-1 creation has been completed. >2014-10-23 13:32:35,205 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-4) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddVm completed, handling the result. >2014-10-23 13:32:35,205 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-4) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddVm succeeded, clearing tasks. >2014-10-23 13:32:35,205 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-4) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 49f15a0a-7877-415b-b236-197923e09c37 >2014-10-23 13:32:35,205 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-4) START, SPMClearTaskVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, taskId = 49f15a0a-7877-415b-b236-197923e09c37), log id: 1103b969 >2014-10-23 13:32:35,206 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-1) START, SetVmStatusVDSCommand( vmId = 71f61d0d-8bec-41d7-86d8-e1db5ea3ab86, status = Down, exit status = Normal), log id: 6c864715 >2014-10-23 13:32:35,207 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-4) START, HSMClearTaskVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, taskId=49f15a0a-7877-415b-b236-197923e09c37), log id: b81a539 >2014-10-23 13:32:35,210 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-1) FINISH, SetVmStatusVDSCommand, log id: 6c864715 >2014-10-23 13:32:35,211 INFO [org.ovirt.engine.core.bll.AddVmCommand] (org.ovirt.thread.pool-7-thread-1) Lock freed to object EngineLock [exclusiveLocks= key: test_pool-2 value: VM_NAME >, sharedLocks= key: 9a68745c-3578-46d7-ad9c-d9e03c4639dd value: DISK >key: 30bad43f-c797-4445-b598-a26dbb2a3400 value: TEMPLATE >] >2014-10-23 13:32:35,216 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-1) Correlation ID: 13519c43, Call Stack: null, Custom Event ID: -1, Message: VM test_pool-2 creation has been completed. >2014-10-23 13:32:35,217 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-1) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddVm completed, handling the result. >2014-10-23 13:32:35,217 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-1) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddVm succeeded, clearing tasks. >2014-10-23 13:32:35,217 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-1) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 7703a2c1-1db8-48b7-8bbf-dafa29596fa4 >2014-10-23 13:32:35,217 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-1) START, SPMClearTaskVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, taskId = 7703a2c1-1db8-48b7-8bbf-dafa29596fa4), log id: 2a154946 >2014-10-23 13:32:35,256 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-4) FINISH, HSMClearTaskVDSCommand, log id: b81a539 >2014-10-23 13:32:35,256 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-4) FINISH, SPMClearTaskVDSCommand, log id: 1103b969 >2014-10-23 13:32:35,258 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-1) START, HSMClearTaskVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, taskId=7703a2c1-1db8-48b7-8bbf-dafa29596fa4), log id: 5b7da83f >2014-10-23 13:32:35,260 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-4) BaseAsyncTask::removeTaskFromDB: Removed task 49f15a0a-7877-415b-b236-197923e09c37 from DataBase >2014-10-23 13:32:35,260 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-4) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 44c122f4-458c-4788-8b9b-e9d5b5afb008 >2014-10-23 13:32:35,306 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-1) FINISH, HSMClearTaskVDSCommand, log id: 5b7da83f >2014-10-23 13:32:35,307 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-1) FINISH, SPMClearTaskVDSCommand, log id: 2a154946 >2014-10-23 13:32:35,312 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-1) BaseAsyncTask::removeTaskFromDB: Removed task 7703a2c1-1db8-48b7-8bbf-dafa29596fa4 from DataBase >2014-10-23 13:32:35,312 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-1) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 7794c7ea-85b1-4fa4-a0d7-28bd6cad32c5 >2014-10-23 13:32:40,048 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp-/127.0.0.1:8702-8) [33274433] Lock Acquired to object EngineLock [exclusiveLocks= key: 2ade22ff-8148-4126-b611-957e920ca7c6 value: VM >, sharedLocks= ] >2014-10-23 13:32:40,090 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp-/127.0.0.1:8702-8) [33274433] START, IsVmDuringInitiatingVDSCommand( vmId = 2ade22ff-8148-4126-b611-957e920ca7c6), log id: 1567f0ea >2014-10-23 13:32:40,090 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp-/127.0.0.1:8702-8) [33274433] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 1567f0ea >2014-10-23 13:32:40,211 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-7-thread-25) [33274433] Running command: RunVmCommand internal: false. Entities affected : ID: 2ade22ff-8148-4126-b611-957e920ca7c6 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-10-23 13:32:40,273 INFO [org.ovirt.engine.core.bll.scheduling.policyunits.HaReservationWeightPolicyUnit] (org.ovirt.thread.pool-7-thread-25) [33274433] Started HA reservation scoring method >2014-10-23 13:32:40,300 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-7-thread-25) [33274433] START, UpdateVmDynamicDataVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@a9d490d2), log id: 26b52e30 >2014-10-23 13:32:40,303 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-7-thread-25) [33274433] FINISH, UpdateVmDynamicDataVDSCommand, log id: 26b52e30 >2014-10-23 13:32:40,319 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-7-thread-25) [33274433] START, CreateVmVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vmId=2ade22ff-8148-4126-b611-957e920ca7c6, vm=VM [test_pool-1]), log id: 7a1123ea >2014-10-23 13:32:40,339 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-7-thread-25) [33274433] START, CreateVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vmId=2ade22ff-8148-4126-b611-957e920ca7c6, vm=VM [test_pool-1]), log id: 16604a6e >2014-10-23 13:32:40,443 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-7-thread-25) [33274433] org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand kvmEnable=true,keyboardLayout=en-us,nice=0,displayNetwork=rhevm,copyPasteEnable=true,timeOffset=0,transparentHugePages=true,vmId=2ade22ff-8148-4126-b611-957e920ca7c6,acpiEnable=true,custom={},spiceSslCipherSuite=DEFAULT,memSize=1024,smp=1,emulatedMachine=rhel6.5.0,vmType=kvm,memGuaranteedSize=1024,display=qxl,smartcardEnable=false,bootMenuEnable=false,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,smpCoresPerSocket=1,maxVCpus=16,devices=[{specParams={vram=32768, heads=1}, device=qxl, type=video, deviceId=1a009684-a482-4a31-a18f-56c3314c6917}, {shared=false, iface=ide, index=2, specParams={path=}, path=, device=cdrom, type=disk, readonly=true, deviceId=44b50e7c-ce5a-4138-8dcc-0ef42ff05f86}, {shared=false, index=0, volumeID=19384420-f743-488a-a94e-0207b6170453, propagateErrors=off, format=cow, type=disk, iface=virtio, bootOrder=1, domainID=a1341057-9cf8-4750-a734-656d05a5d2c5, imageID=4c624442-cb01-4403-a4e0-d581d544a584, specParams={}, optional=false, device=disk, poolID=f8f5eaee-8fd0-4b45-87db-62d61b03a916, readonly=false, deviceId=4c624442-cb01-4403-a4e0-d581d544a584}, {nicModel=pv, specParams={outbound={}, inbound={}}, macAddr=00:1a:4a:91:91:08, device=bridge, linkActive=true, type=interface, filter=vdsm-no-mac-spoofing, network=rhevm, deviceId=859e13b3-da33-4d4b-9626-9e08a8ea8723}, {specParams={model=virtio}, device=memballoon, type=balloon, deviceId=06fcb479-7202-4fa8-9c2c-3a0fb9a664af}, {index=0, model=virtio-scsi, specParams={}, device=scsi, type=controller, deviceId=c4a5b8c1-0cdc-48fd-8264-e2ac495396c5}, {specParams={}, device=virtio-serial, type=controller, deviceId=7432f855-6b1f-4c51-aa3b-26951869f79c}],vmName=test_pool-1,cpuType=Opteron_G1,fileTransferEnable=true >2014-10-23 13:32:40,501 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-7-thread-25) [33274433] FINISH, CreateVDSCommand, log id: 16604a6e >2014-10-23 13:32:40,510 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-7-thread-25) [33274433] FINISH, CreateVmVDSCommand, return: WaitForLaunch, log id: 7a1123ea >2014-10-23 13:32:40,511 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-7-thread-25) [33274433] Lock freed to object EngineLock [exclusiveLocks= key: 2ade22ff-8148-4126-b611-957e920ca7c6 value: VM >, sharedLocks= ] >2014-10-23 13:32:40,525 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-25) [33274433] Correlation ID: 33274433, Job ID: 310f7769-f5fd-44e3-ad82-d4946ff4dfcb, Call Stack: null, Custom Event ID: -1, Message: VM test_pool-1 was started by admin (Host: alma06.qa.lab.tlv.redhat.com). >2014-10-23 13:32:42,201 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-31) VM test_pool-1 2ade22ff-8148-4126-b611-957e920ca7c6 moved from WaitForLaunch --> PoweringUp >2014-10-23 13:32:42,203 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-31) START, FullListVdsCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c], vmIds=[2ade22ff-8148-4126-b611-957e920ca7c6]), log id: 9f4e92b >2014-10-23 13:32:42,216 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-31) FINISH, FullListVdsCommand, return: [{acpiEnable=true, emulatedMachine=rhel6.5.0, pid=25428, memGuaranteedSize=1024, transparentHugePages=true, displaySecurePort=5903, spiceSslCipherSuite=DEFAULT, cpuType=Opteron_G1, smp=1, pauseCode=NOERR, custom={}, vmType=kvm, memSize=1024, smpCoresPerSocket=1, vmName=test_pool-1, nice=0, status=Up, bootMenuEnable=false, vmId=2ade22ff-8148-4126-b611-957e920ca7c6, copyPasteEnable=true, displayIp=10.35.117.30, displayPort=5902, smartcardEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, fileTransferEnable=true, nicModel=rtl8139,pv, keyboardLayout=en-us, kvmEnable=true, displayNetwork=rhevm, devices=[Ljava.lang.Object;@2df80b28, timeOffset=0, maxVCpus=16, clientIp=, display=qxl}], log id: 9f4e92b >2014-10-23 13:32:42,219 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-31) Received a spice Device without an address when processing VM 2ade22ff-8148-4126-b611-957e920ca7c6 devices, skipping device: {tlsPort=5903, specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, copyPasteEnable=true, keyMap=en-us, displayIp=10.35.117.30, displayNetwork=rhevm}, deviceType=graphics, device=spice, type=graphics, port=5902} >2014-10-23 13:32:46,363 INFO [org.ovirt.engine.core.bll.ShutdownVmCommand] (org.ovirt.thread.pool-7-thread-8) [1f4111d3] Running command: ShutdownVmCommand internal: false. Entities affected : ID: 2ade22ff-8148-4126-b611-957e920ca7c6 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-10-23 13:32:46,367 INFO [org.ovirt.engine.core.bll.ShutdownVmCommand] (org.ovirt.thread.pool-7-thread-8) [1f4111d3] Entered (VM test_pool-1). >2014-10-23 13:32:46,367 INFO [org.ovirt.engine.core.bll.ShutdownVmCommand] (org.ovirt.thread.pool-7-thread-8) [1f4111d3] Cannot shutdown VM test_pool-1, status is not up. Stopping instead. >2014-10-23 13:32:46,387 INFO [org.ovirt.engine.core.bll.StopVmCommand] (org.ovirt.thread.pool-7-thread-8) [2999119c] Running command: StopVmCommand internal: true. Entities affected : ID: 2ade22ff-8148-4126-b611-957e920ca7c6 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-10-23 13:32:46,393 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (org.ovirt.thread.pool-7-thread-8) [2999119c] START, DestroyVmVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vmId=2ade22ff-8148-4126-b611-957e920ca7c6, force=false, secondsToWait=0, gracefully=false, reason=), log id: 3bbac9c >2014-10-23 13:32:46,406 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-7-thread-8) [2999119c] START, DestroyVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vmId=2ade22ff-8148-4126-b611-957e920ca7c6, force=false, secondsToWait=0, gracefully=false, reason=), log id: 669b805f >2014-10-23 13:32:47,044 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-7-thread-8) [2999119c] FINISH, DestroyVDSCommand, log id: 669b805f >2014-10-23 13:32:47,054 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (org.ovirt.thread.pool-7-thread-8) [2999119c] FINISH, DestroyVmVDSCommand, return: Down, log id: 3bbac9c >2014-10-23 13:32:47,080 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (org.ovirt.thread.pool-7-thread-2) [56cb1e56] Running command: ProcessDownVmCommand internal: true. >2014-10-23 13:32:47,097 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-8) [2999119c] Correlation ID: 2999119c, Job ID: 2afda1b0-0796-47bf-9d22-383651d15646, Call Stack: null, Custom Event ID: -1, Message: VM test_pool-1 was powered off ungracefully by admin (Host: alma06.qa.lab.tlv.redhat.com) (Reason: Not Specified). >2014-10-23 13:32:58,198 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-7-thread-49) [44ece1cc] Start running CanDoAction for command number 1/2 (Command type: RemoveVmFromPool) >2014-10-23 13:32:58,198 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-7-thread-32) [4157c4e1] Start running CanDoAction for command number 2/2 (Command type: RemoveVmFromPool) >2014-10-23 13:32:58,215 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-7-thread-49) [44ece1cc] Finish handling CanDoAction for command number 1/2 (Command type: RemoveVmFromPool) >2014-10-23 13:32:58,217 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-7-thread-32) [4157c4e1] Finish handling CanDoAction for command number 2/2 (Command type: RemoveVmFromPool) >2014-10-23 13:32:58,219 INFO [org.ovirt.engine.core.bll.RemoveVmFromPoolCommand] (org.ovirt.thread.pool-7-thread-36) [44ece1cc] Running command: RemoveVmFromPoolCommand internal: false. Entities affected : ID: 71384dea-9dfe-4be4-b962-72e4ad0d64d0 Type: VmPoolAction group EDIT_VM_POOL_CONFIGURATION with role type USER >2014-10-23 13:32:58,254 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-36) [44ece1cc] Correlation ID: 44ece1cc, Call Stack: null, Custom Event ID: -1, Message: VM test_pool-1 was removed from VM Pool test_pool by admin. >2014-10-23 13:32:58,254 INFO [org.ovirt.engine.core.bll.RemoveVmFromPoolCommand] (org.ovirt.thread.pool-7-thread-36) [4157c4e1] Running command: RemoveVmFromPoolCommand internal: false. Entities affected : ID: 71384dea-9dfe-4be4-b962-72e4ad0d64d0 Type: VmPoolAction group EDIT_VM_POOL_CONFIGURATION with role type USER >2014-10-23 13:32:58,265 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-36) [4157c4e1] Correlation ID: 4157c4e1, Call Stack: null, Custom Event ID: -1, Message: VM test_pool-2 was removed from VM Pool test_pool by admin. >2014-10-23 13:32:58,309 INFO [org.ovirt.engine.core.bll.RemoveVmPoolCommand] (org.ovirt.thread.pool-7-thread-36) [2494213] Running command: RemoveVmPoolCommand internal: true. Entities affected : ID: 71384dea-9dfe-4be4-b962-72e4ad0d64d0 Type: VmPoolAction group DELETE_VM_POOL with role type USER >2014-10-23 13:32:58,337 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-36) [2494213] Correlation ID: 2494213, Call Stack: null, Custom Event ID: -1, Message: VM Pool test_pool was removed by admin. >2014-10-23 13:33:06,189 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-7-thread-11) [2c38239e] Start running CanDoAction for command number 2/2 (Command type: RemoveVm) >2014-10-23 13:33:06,189 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-7-thread-14) [2565048a] Start running CanDoAction for command number 1/2 (Command type: RemoveVm) >2014-10-23 13:33:06,191 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-11) [2c38239e] Lock Acquired to object EngineLock [exclusiveLocks= key: 71f61d0d-8bec-41d7-86d8-e1db5ea3ab86 value: VM >, sharedLocks= ] >2014-10-23 13:33:06,191 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-14) [2565048a] Lock Acquired to object EngineLock [exclusiveLocks= key: 2ade22ff-8148-4126-b611-957e920ca7c6 value: VM >, sharedLocks= ] >2014-10-23 13:33:06,213 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-7-thread-11) [2c38239e] Finish handling CanDoAction for command number 2/2 (Command type: RemoveVm) >2014-10-23 13:33:06,228 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-7-thread-14) [2565048a] Finish handling CanDoAction for command number 1/2 (Command type: RemoveVm) >2014-10-23 13:33:06,245 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-37) [2565048a] Running command: RemoveVmCommand internal: false. Entities affected : ID: 2ade22ff-8148-4126-b611-957e920ca7c6 Type: VMAction group DELETE_VM with role type USER >2014-10-23 13:33:06,245 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-37) [2565048a] START, SetVmStatusVDSCommand( vmId = 2ade22ff-8148-4126-b611-957e920ca7c6, status = ImageLocked, exit status = Normal), log id: 3ea1cc08 >2014-10-23 13:33:06,249 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-37) [2565048a] FINISH, SetVmStatusVDSCommand, log id: 3ea1cc08 >2014-10-23 13:33:06,253 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-37) [2565048a] Lock freed to object EngineLock [exclusiveLocks= key: 2ade22ff-8148-4126-b611-957e920ca7c6 value: VM >, sharedLocks= ] >2014-10-23 13:33:06,309 INFO [org.ovirt.engine.core.bll.RemoveAllVmImagesCommand] (org.ovirt.thread.pool-7-thread-37) [1ceb32f5] Running command: RemoveAllVmImagesCommand internal: true. Entities affected : ID: 2ade22ff-8148-4126-b611-957e920ca7c6 Type: VM >2014-10-23 13:33:06,314 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-7-thread-37) [1adc2c91] Running command: RemoveImageCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage >2014-10-23 13:33:06,333 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-7-thread-37) [1adc2c91] START, DeleteImageGroupVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = 4c624442-cb01-4403-a4e0-d581d544a584, postZeros = false, forceDelete = false), log id: 5b2d9701 >2014-10-23 13:33:06,505 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-7-thread-37) [1adc2c91] FINISH, DeleteImageGroupVDSCommand, log id: 5b2d9701 >2014-10-23 13:33:06,514 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-37) [1adc2c91] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command f34b7b22-f3cf-4cab-a229-d06fec05957a >2014-10-23 13:33:06,514 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-7-thread-37) [1adc2c91] CommandMultiAsyncTasks::AttachTask: Attaching task f4404847-ccf8-40aa-a4ba-324f36ee6809 to command f34b7b22-f3cf-4cab-a229-d06fec05957a. >2014-10-23 13:33:06,525 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-7-thread-37) [1adc2c91] Adding task f4404847-ccf8-40aa-a4ba-324f36ee6809 (Parent Command RemoveVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-10-23 13:33:06,669 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-37) [1adc2c91] Correlation ID: 2565048a, Job ID: 10367b9f-55cf-416d-b57d-4dfbb9f18e2b, Call Stack: null, Custom Event ID: -1, Message: VM test_pool-1 was successfully removed. >2014-10-23 13:33:06,669 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-37) [1adc2c91] BaseAsyncTask::startPollingTask: Starting to poll task f4404847-ccf8-40aa-a4ba-324f36ee6809. >2014-10-23 13:33:06,685 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-37) [2c38239e] Running command: RemoveVmCommand internal: false. Entities affected : ID: 71f61d0d-8bec-41d7-86d8-e1db5ea3ab86 Type: VMAction group DELETE_VM with role type USER >2014-10-23 13:33:06,686 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-37) [2c38239e] START, SetVmStatusVDSCommand( vmId = 71f61d0d-8bec-41d7-86d8-e1db5ea3ab86, status = ImageLocked, exit status = Normal), log id: 7cf4f0b8 >2014-10-23 13:33:06,690 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-37) [2c38239e] FINISH, SetVmStatusVDSCommand, log id: 7cf4f0b8 >2014-10-23 13:33:06,692 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-37) [2c38239e] Lock freed to object EngineLock [exclusiveLocks= key: 71f61d0d-8bec-41d7-86d8-e1db5ea3ab86 value: VM >, sharedLocks= ] >2014-10-23 13:33:06,721 INFO [org.ovirt.engine.core.bll.RemoveAllVmImagesCommand] (org.ovirt.thread.pool-7-thread-37) [3e79f2ad] Running command: RemoveAllVmImagesCommand internal: true. Entities affected : ID: 71f61d0d-8bec-41d7-86d8-e1db5ea3ab86 Type: VM >2014-10-23 13:33:06,736 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-7-thread-37) [29d225d4] Running command: RemoveImageCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage >2014-10-23 13:33:06,758 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-7-thread-37) [29d225d4] START, DeleteImageGroupVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = ddf27c13-9931-4730-ba37-33f1325c47c0, postZeros = false, forceDelete = false), log id: 44217d04 >2014-10-23 13:33:07,122 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-7-thread-37) [29d225d4] FINISH, DeleteImageGroupVDSCommand, log id: 44217d04 >2014-10-23 13:33:07,127 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-37) [29d225d4] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 97ae4acf-51d2-44c8-a954-00a13b9c9e5c >2014-10-23 13:33:07,127 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-7-thread-37) [29d225d4] CommandMultiAsyncTasks::AttachTask: Attaching task 11e1ff3c-87f1-40d9-b66b-ed7fe11b8eea to command 97ae4acf-51d2-44c8-a954-00a13b9c9e5c. >2014-10-23 13:33:07,137 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-7-thread-37) [29d225d4] Adding task 11e1ff3c-87f1-40d9-b66b-ed7fe11b8eea (Parent Command RemoveVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-10-23 13:33:07,192 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-37) [29d225d4] Correlation ID: 2c38239e, Job ID: e2a8adb1-8ac0-4463-acba-21e0322eff8b, Call Stack: null, Custom Event ID: -1, Message: VM test_pool-2 was successfully removed. >2014-10-23 13:33:07,192 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-37) [29d225d4] BaseAsyncTask::startPollingTask: Starting to poll task 11e1ff3c-87f1-40d9-b66b-ed7fe11b8eea. >2014-10-23 13:33:15,058 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-4) Polling and updating Async Tasks: 4 tasks, 2 tasks to poll now >2014-10-23 13:33:15,079 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-4) SPMAsyncTask::PollTask: Polling task 11e1ff3c-87f1-40d9-b66b-ed7fe11b8eea (Parent Command RemoveVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-10-23 13:33:15,085 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-4) BaseAsyncTask::onTaskEndSuccess: Task 11e1ff3c-87f1-40d9-b66b-ed7fe11b8eea (Parent Command RemoveVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-10-23 13:33:15,086 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-4) CommandAsyncTask::endActionIfNecessary: All tasks of command 97ae4acf-51d2-44c8-a954-00a13b9c9e5c has ended -> executing endAction >2014-10-23 13:33:15,086 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-4) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: 97ae4acf-51d2-44c8-a954-00a13b9c9e5c): calling endAction . >2014-10-23 13:33:15,086 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-4) SPMAsyncTask::PollTask: Polling task f4404847-ccf8-40aa-a4ba-324f36ee6809 (Parent Command RemoveVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-10-23 13:33:15,087 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-35) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction RemoveVm, executionIndex: 0 >2014-10-23 13:33:15,093 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-4) BaseAsyncTask::onTaskEndSuccess: Task f4404847-ccf8-40aa-a4ba-324f36ee6809 (Parent Command RemoveVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-10-23 13:33:15,096 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-4) CommandAsyncTask::endActionIfNecessary: All tasks of command f34b7b22-f3cf-4cab-a229-d06fec05957a has ended -> executing endAction >2014-10-23 13:33:15,096 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-4) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: f34b7b22-f3cf-4cab-a229-d06fec05957a): calling endAction . >2014-10-23 13:33:15,098 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-45) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction RemoveVm, executionIndex: 0 >2014-10-23 13:33:15,119 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-45) Ending command successfully: org.ovirt.engine.core.bll.RemoveVmCommand >2014-10-23 13:33:15,120 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-45) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RemoveVm completed, handling the result. >2014-10-23 13:33:15,120 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-45) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RemoveVm succeeded, clearing tasks. >2014-10-23 13:33:15,123 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-35) Ending command successfully: org.ovirt.engine.core.bll.RemoveVmCommand >2014-10-23 13:33:15,124 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-35) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RemoveVm completed, handling the result. >2014-10-23 13:33:15,124 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-35) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RemoveVm succeeded, clearing tasks. >2014-10-23 13:33:15,126 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-35) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 11e1ff3c-87f1-40d9-b66b-ed7fe11b8eea >2014-10-23 13:33:15,127 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-35) START, SPMClearTaskVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, taskId = 11e1ff3c-87f1-40d9-b66b-ed7fe11b8eea), log id: 6694869b >2014-10-23 13:33:15,134 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-35) START, HSMClearTaskVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, taskId=11e1ff3c-87f1-40d9-b66b-ed7fe11b8eea), log id: 21115eba >2014-10-23 13:33:15,141 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-45) SPMAsyncTask::ClearAsyncTask: Attempting to clear task f4404847-ccf8-40aa-a4ba-324f36ee6809 >2014-10-23 13:33:15,141 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-45) START, SPMClearTaskVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, taskId = f4404847-ccf8-40aa-a4ba-324f36ee6809), log id: 743d1903 >2014-10-23 13:33:15,160 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-35) FINISH, HSMClearTaskVDSCommand, log id: 21115eba >2014-10-23 13:33:15,160 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-35) FINISH, SPMClearTaskVDSCommand, log id: 6694869b >2014-10-23 13:33:15,162 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-45) START, HSMClearTaskVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, taskId=f4404847-ccf8-40aa-a4ba-324f36ee6809), log id: 3c785923 >2014-10-23 13:33:15,166 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-35) BaseAsyncTask::removeTaskFromDB: Removed task 11e1ff3c-87f1-40d9-b66b-ed7fe11b8eea from DataBase >2014-10-23 13:33:15,166 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-35) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 97ae4acf-51d2-44c8-a954-00a13b9c9e5c >2014-10-23 13:33:15,190 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-45) FINISH, HSMClearTaskVDSCommand, log id: 3c785923 >2014-10-23 13:33:15,190 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-45) FINISH, SPMClearTaskVDSCommand, log id: 743d1903 >2014-10-23 13:33:15,195 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-45) BaseAsyncTask::removeTaskFromDB: Removed task f4404847-ccf8-40aa-a4ba-324f36ee6809 from DataBase >2014-10-23 13:33:15,195 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-45) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity f34b7b22-f3cf-4cab-a229-d06fec05957a >2014-10-23 13:33:15,260 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-7-thread-26) [778af3f0] Start running CanDoAction for command number 1/2 (Command type: RemoveVm) >2014-10-23 13:33:15,260 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-7-thread-24) [59dca6c4] Start running CanDoAction for command number 2/2 (Command type: RemoveVm) >2014-10-23 13:33:15,262 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-24) [59dca6c4] Lock Acquired to object EngineLock [exclusiveLocks= key: 9236151d-8fb4-4c9a-830a-53af2b584dc7 value: VM >, sharedLocks= ] >2014-10-23 13:33:15,275 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-26) [778af3f0] Lock Acquired to object EngineLock [exclusiveLocks= key: 3e13f009-03f3-4a43-b540-ad7fe3122e53 value: VM >, sharedLocks= ] >2014-10-23 13:33:15,286 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-7-thread-24) [59dca6c4] Finish handling CanDoAction for command number 2/2 (Command type: RemoveVm) >2014-10-23 13:33:15,331 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-7-thread-26) [778af3f0] Finish handling CanDoAction for command number 1/2 (Command type: RemoveVm) >2014-10-23 13:33:15,353 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-12) [778af3f0] Running command: RemoveVmCommand internal: false. Entities affected : ID: 3e13f009-03f3-4a43-b540-ad7fe3122e53 Type: VMAction group DELETE_VM with role type USER >2014-10-23 13:33:15,353 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-12) [778af3f0] START, SetVmStatusVDSCommand( vmId = 3e13f009-03f3-4a43-b540-ad7fe3122e53, status = ImageLocked, exit status = Normal), log id: 46b48c04 >2014-10-23 13:33:15,358 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-12) [778af3f0] FINISH, SetVmStatusVDSCommand, log id: 46b48c04 >2014-10-23 13:33:15,360 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-12) [778af3f0] Lock freed to object EngineLock [exclusiveLocks= key: 3e13f009-03f3-4a43-b540-ad7fe3122e53 value: VM >, sharedLocks= ] >2014-10-23 13:33:15,383 INFO [org.ovirt.engine.core.bll.RemoveAllVmImagesCommand] (org.ovirt.thread.pool-7-thread-12) [61617163] Running command: RemoveAllVmImagesCommand internal: true. Entities affected : ID: 3e13f009-03f3-4a43-b540-ad7fe3122e53 Type: VM >2014-10-23 13:33:15,388 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-7-thread-12) [7d55b357] Running command: RemoveImageCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage >2014-10-23 13:33:15,414 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-7-thread-12) [7d55b357] START, DeleteImageGroupVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = e3162e04-764d-40f0-993c-34865796c725, postZeros = false, forceDelete = false), log id: 5caa8e65 >2014-10-23 13:33:15,731 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-7-thread-12) [7d55b357] FINISH, DeleteImageGroupVDSCommand, log id: 5caa8e65 >2014-10-23 13:33:15,738 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-12) [7d55b357] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 9c8228fa-c696-4ea0-afef-d730009b12cb >2014-10-23 13:33:15,738 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-7-thread-12) [7d55b357] CommandMultiAsyncTasks::AttachTask: Attaching task 1b5e7cce-67dc-491e-85b5-7ca000bb840e to command 9c8228fa-c696-4ea0-afef-d730009b12cb. >2014-10-23 13:33:15,748 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-7-thread-12) [7d55b357] Adding task 1b5e7cce-67dc-491e-85b5-7ca000bb840e (Parent Command RemoveVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-10-23 13:33:16,049 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-12) [7d55b357] Correlation ID: 778af3f0, Job ID: d18ccd2c-05dd-4827-8805-2c3e83068cd6, Call Stack: null, Custom Event ID: -1, Message: VM baloon_vm was successfully removed. >2014-10-23 13:33:16,049 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-12) [7d55b357] BaseAsyncTask::startPollingTask: Starting to poll task 1b5e7cce-67dc-491e-85b5-7ca000bb840e. >2014-10-23 13:33:16,109 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-12) [59dca6c4] Running command: RemoveVmCommand internal: false. Entities affected : ID: 9236151d-8fb4-4c9a-830a-53af2b584dc7 Type: VMAction group DELETE_VM with role type USER >2014-10-23 13:33:16,110 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-12) [59dca6c4] START, SetVmStatusVDSCommand( vmId = 9236151d-8fb4-4c9a-830a-53af2b584dc7, status = ImageLocked, exit status = Normal), log id: 7ce9acf5 >2014-10-23 13:33:16,113 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-12) [59dca6c4] FINISH, SetVmStatusVDSCommand, log id: 7ce9acf5 >2014-10-23 13:33:16,115 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-12) [59dca6c4] Lock freed to object EngineLock [exclusiveLocks= key: 9236151d-8fb4-4c9a-830a-53af2b584dc7 value: VM >, sharedLocks= ] >2014-10-23 13:33:16,136 INFO [org.ovirt.engine.core.bll.RemoveAllVmImagesCommand] (org.ovirt.thread.pool-7-thread-12) [43cad946] Running command: RemoveAllVmImagesCommand internal: true. Entities affected : ID: 9236151d-8fb4-4c9a-830a-53af2b584dc7 Type: VM >2014-10-23 13:33:16,139 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-7-thread-12) [1d3a9ad3] Running command: RemoveImageCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage >2014-10-23 13:33:16,164 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-7-thread-12) [1d3a9ad3] START, DeleteImageGroupVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = 493ae1d9-5bb7-4948-a637-3e5bc861ac45, postZeros = false, forceDelete = false), log id: 6a9898e9 >2014-10-23 13:33:16,320 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-7-thread-12) [1d3a9ad3] FINISH, DeleteImageGroupVDSCommand, log id: 6a9898e9 >2014-10-23 13:33:16,325 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-12) [1d3a9ad3] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 74585241-cce5-45fa-80ed-5ecf7cc00107 >2014-10-23 13:33:16,325 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-7-thread-12) [1d3a9ad3] CommandMultiAsyncTasks::AttachTask: Attaching task 19e6df27-25b9-4402-a32b-ead1566977d2 to command 74585241-cce5-45fa-80ed-5ecf7cc00107. >2014-10-23 13:33:16,334 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-7-thread-12) [1d3a9ad3] Adding task 19e6df27-25b9-4402-a32b-ead1566977d2 (Parent Command RemoveVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-10-23 13:33:16,525 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-12) [1d3a9ad3] Correlation ID: 59dca6c4, Job ID: df01c746-b691-4661-b031-9163923a6492, Call Stack: null, Custom Event ID: -1, Message: VM test_vm was successfully removed. >2014-10-23 13:33:16,525 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-12) [1d3a9ad3] BaseAsyncTask::startPollingTask: Starting to poll task 19e6df27-25b9-4402-a32b-ead1566977d2. >2014-10-23 13:33:25,097 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-2) Polling and updating Async Tasks: 6 tasks, 2 tasks to poll now >2014-10-23 13:33:25,108 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-2) SPMAsyncTask::PollTask: Polling task 19e6df27-25b9-4402-a32b-ead1566977d2 (Parent Command RemoveVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-10-23 13:33:25,116 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-2) BaseAsyncTask::onTaskEndSuccess: Task 19e6df27-25b9-4402-a32b-ead1566977d2 (Parent Command RemoveVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-10-23 13:33:25,122 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-2) CommandAsyncTask::endActionIfNecessary: All tasks of command 74585241-cce5-45fa-80ed-5ecf7cc00107 has ended -> executing endAction >2014-10-23 13:33:25,122 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-2) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: 74585241-cce5-45fa-80ed-5ecf7cc00107): calling endAction . >2014-10-23 13:33:25,122 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-2) SPMAsyncTask::PollTask: Polling task 1b5e7cce-67dc-491e-85b5-7ca000bb840e (Parent Command RemoveVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-10-23 13:33:25,123 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-50) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction RemoveVm, executionIndex: 0 >2014-10-23 13:33:25,129 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-2) BaseAsyncTask::onTaskEndSuccess: Task 1b5e7cce-67dc-491e-85b5-7ca000bb840e (Parent Command RemoveVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-10-23 13:33:25,130 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-2) CommandAsyncTask::endActionIfNecessary: All tasks of command 9c8228fa-c696-4ea0-afef-d730009b12cb has ended -> executing endAction >2014-10-23 13:33:25,130 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-2) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: 9c8228fa-c696-4ea0-afef-d730009b12cb): calling endAction . >2014-10-23 13:33:25,130 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-2) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction RemoveVm, executionIndex: 0 >2014-10-23 13:33:25,144 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-50) Ending command successfully: org.ovirt.engine.core.bll.RemoveVmCommand >2014-10-23 13:33:25,146 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-50) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RemoveVm completed, handling the result. >2014-10-23 13:33:25,146 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-50) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RemoveVm succeeded, clearing tasks. >2014-10-23 13:33:25,146 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-7-thread-2) Ending command successfully: org.ovirt.engine.core.bll.RemoveVmCommand >2014-10-23 13:33:25,147 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-2) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RemoveVm completed, handling the result. >2014-10-23 13:33:25,147 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-2) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RemoveVm succeeded, clearing tasks. >2014-10-23 13:33:25,150 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-2) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 1b5e7cce-67dc-491e-85b5-7ca000bb840e >2014-10-23 13:33:25,151 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-2) START, SPMClearTaskVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, taskId = 1b5e7cce-67dc-491e-85b5-7ca000bb840e), log id: 4f2b89f4 >2014-10-23 13:33:25,153 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-2) START, HSMClearTaskVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, taskId=1b5e7cce-67dc-491e-85b5-7ca000bb840e), log id: 336e1ac7 >2014-10-23 13:33:25,153 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-50) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 19e6df27-25b9-4402-a32b-ead1566977d2 >2014-10-23 13:33:25,154 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-50) START, SPMClearTaskVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, taskId = 19e6df27-25b9-4402-a32b-ead1566977d2), log id: 6a8b5229 >2014-10-23 13:33:25,181 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-2) FINISH, HSMClearTaskVDSCommand, log id: 336e1ac7 >2014-10-23 13:33:25,181 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-2) FINISH, SPMClearTaskVDSCommand, log id: 4f2b89f4 >2014-10-23 13:33:25,183 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-50) START, HSMClearTaskVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, taskId=19e6df27-25b9-4402-a32b-ead1566977d2), log id: 60bf3864 >2014-10-23 13:33:25,185 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-2) BaseAsyncTask::removeTaskFromDB: Removed task 1b5e7cce-67dc-491e-85b5-7ca000bb840e from DataBase >2014-10-23 13:33:25,185 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-2) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 9c8228fa-c696-4ea0-afef-d730009b12cb >2014-10-23 13:33:25,209 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-50) FINISH, HSMClearTaskVDSCommand, log id: 60bf3864 >2014-10-23 13:33:25,209 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-50) FINISH, SPMClearTaskVDSCommand, log id: 6a8b5229 >2014-10-23 13:33:25,212 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-50) BaseAsyncTask::removeTaskFromDB: Removed task 19e6df27-25b9-4402-a32b-ead1566977d2 from DataBase >2014-10-23 13:33:25,212 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-50) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 74585241-cce5-45fa-80ed-5ecf7cc00107 >2014-10-23 13:33:50,074 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-44) Setting new tasks map. The map contains now 4 tasks >2014-10-23 13:34:20,075 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-99) Setting new tasks map. The map contains now 2 tasks >2014-10-23 13:34:50,076 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-56) Setting new tasks map. The map contains now 0 tasks >2014-10-23 13:34:50,077 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-56) Cleared all tasks of pool f8f5eaee-8fd0-4b45-87db-62d61b03a916. >2014-10-23 13:35:23,262 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-68) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-23 13:35:31,562 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-43) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-23 13:54:54,565 INFO [org.ovirt.engine.core.bll.AddVmFromScratchCommand] (ajp-/127.0.0.1:8702-5) [284121b3] Lock Acquired to object EngineLock [exclusiveLocks= key: test_memory_vm value: VM_NAME >, sharedLocks= ] >2014-10-23 13:54:54,610 INFO [org.ovirt.engine.core.bll.AddVmFromScratchCommand] (ajp-/127.0.0.1:8702-5) [284121b3] Running command: AddVmFromScratchCommand internal: false. Entities affected : ID: 08ea0169-ae42-4e1b-bb20-839a66cd2010 Type: VdsGroupsAction group CREATE_VM with role type USER >2014-10-23 13:54:54,799 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [284121b3] Correlation ID: 284121b3, Job ID: e2a36952-3279-4f7b-9df7-d3bd7110fa5f, Call Stack: null, Custom Event ID: -1, Message: VM test_memory_vm was created by admin. >2014-10-23 13:54:54,805 INFO [org.ovirt.engine.core.bll.AddVmFromScratchCommand] (ajp-/127.0.0.1:8702-5) [284121b3] Lock freed to object EngineLock [exclusiveLocks= key: test_memory_vm value: VM_NAME >, sharedLocks= ] >2014-10-23 13:54:55,690 INFO [org.ovirt.engine.core.bll.network.vm.ReorderVmNicsCommand] (ajp-/127.0.0.1:8702-1) [3442ffc0] Running command: ReorderVmNicsCommand internal: false. Entities affected : ID: 588d43e4-d12e-49b8-b834-734dbcaaf703 Type: VMAction group CREATE_VM with role type USER >2014-10-23 13:55:00,314 INFO [org.ovirt.engine.core.bll.AddDiskCommand] (ajp-/127.0.0.1:8702-9) [5feb8056] Lock Acquired to object EngineLock [exclusiveLocks= key: 588d43e4-d12e-49b8-b834-734dbcaaf703 value: VM_DISK_BOOT >, sharedLocks= key: 588d43e4-d12e-49b8-b834-734dbcaaf703 value: VM >] >2014-10-23 13:55:00,390 INFO [org.ovirt.engine.core.bll.AddDiskCommand] (ajp-/127.0.0.1:8702-9) [5feb8056] Running command: AddDiskCommand internal: false. Entities affected : ID: 588d43e4-d12e-49b8-b834-734dbcaaf703 Type: VMAction group CONFIGURE_VM_STORAGE with role type USER, ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: StorageAction group CREATE_DISK with role type USER >2014-10-23 13:55:00,437 INFO [org.ovirt.engine.core.bll.AddImageFromScratchCommand] (ajp-/127.0.0.1:8702-9) [452d7595] Running command: AddImageFromScratchCommand internal: true. Entities affected : ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: Storage >2014-10-23 13:55:00,448 INFO [org.ovirt.engine.core.bll.AddImageFromScratchCommand] (ajp-/127.0.0.1:8702-9) [452d7595] Lock freed to object EngineLock [exclusiveLocks= key: 588d43e4-d12e-49b8-b834-734dbcaaf703 value: VM_DISK_BOOT >, sharedLocks= key: 588d43e4-d12e-49b8-b834-734dbcaaf703 value: VM >] >2014-10-23 13:55:00,449 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (ajp-/127.0.0.1:8702-9) [452d7595] START, CreateImageVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = 59ed3435-ad92-48ff-be5f-c2c1c89b6fa8, imageSizeInBytes = 1073741824, volumeFormat = RAW, newImageId = 2457fd51-d920-4394-9ce3-4d801ece1a39, newImageDescription = ), log id: 5aadeb72 >2014-10-23 13:55:00,450 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (ajp-/127.0.0.1:8702-9) [452d7595] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-10-23 13:55:00,603 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (ajp-/127.0.0.1:8702-9) [452d7595] FINISH, CreateImageVDSCommand, return: 2457fd51-d920-4394-9ce3-4d801ece1a39, log id: 5aadeb72 >2014-10-23 13:55:00,608 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (ajp-/127.0.0.1:8702-9) [452d7595] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 462f7ae2-7812-4357-8a4c-4956391c8b16 >2014-10-23 13:55:00,609 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (ajp-/127.0.0.1:8702-9) [452d7595] CommandMultiAsyncTasks::AttachTask: Attaching task 544888fc-39f1-4f2c-8557-8a68e9dfe6cf to command 462f7ae2-7812-4357-8a4c-4956391c8b16. >2014-10-23 13:55:00,618 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (ajp-/127.0.0.1:8702-9) [452d7595] Adding task 544888fc-39f1-4f2c-8557-8a68e9dfe6cf (Parent Command AddDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-10-23 13:55:00,670 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [452d7595] Correlation ID: 5feb8056, Job ID: 4c150c65-ac85-43d1-9a24-2fac91d836ae, Call Stack: null, Custom Event ID: -1, Message: Add-Disk operation of test_memory_vm_Disk1 was initiated on VM test_memory_vm by admin. >2014-10-23 13:55:00,670 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (ajp-/127.0.0.1:8702-9) [452d7595] BaseAsyncTask::startPollingTask: Starting to poll task 544888fc-39f1-4f2c-8557-8a68e9dfe6cf. >2014-10-23 13:55:05,183 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-52) Polling and updating Async Tasks: 1 tasks, 1 tasks to poll now >2014-10-23 13:55:05,198 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-52) SPMAsyncTask::PollTask: Polling task 544888fc-39f1-4f2c-8557-8a68e9dfe6cf (Parent Command AddDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-10-23 13:55:05,203 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-52) BaseAsyncTask::onTaskEndSuccess: Task 544888fc-39f1-4f2c-8557-8a68e9dfe6cf (Parent Command AddDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-10-23 13:55:05,204 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-52) CommandAsyncTask::endActionIfNecessary: All tasks of command 462f7ae2-7812-4357-8a4c-4956391c8b16 has ended -> executing endAction >2014-10-23 13:55:05,204 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-52) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: 462f7ae2-7812-4357-8a4c-4956391c8b16): calling endAction . >2014-10-23 13:55:05,204 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-23) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction AddDisk, executionIndex: 0 >2014-10-23 13:55:05,218 INFO [org.ovirt.engine.core.bll.AddDiskCommand] (org.ovirt.thread.pool-7-thread-23) [5feb8056] Ending command successfully: org.ovirt.engine.core.bll.AddDiskCommand >2014-10-23 13:55:05,228 INFO [org.ovirt.engine.core.bll.AddImageFromScratchCommand] (org.ovirt.thread.pool-7-thread-23) [452d7595] Ending command successfully: org.ovirt.engine.core.bll.AddImageFromScratchCommand >2014-10-23 13:55:05,243 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-23) [452d7595] START, GetImageInfoVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = 59ed3435-ad92-48ff-be5f-c2c1c89b6fa8, imageId = 2457fd51-d920-4394-9ce3-4d801ece1a39), log id: 366e4d58 >2014-10-23 13:55:05,291 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-23) [452d7595] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@326de172, log id: 366e4d58 >2014-10-23 13:55:05,306 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-23) Correlation ID: 5feb8056, Job ID: 4c150c65-ac85-43d1-9a24-2fac91d836ae, Call Stack: null, Custom Event ID: -1, Message: The disk test_memory_vm_Disk1 was successfully added to VM test_memory_vm. >2014-10-23 13:55:05,306 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-23) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddDisk completed, handling the result. >2014-10-23 13:55:05,306 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-23) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddDisk succeeded, clearing tasks. >2014-10-23 13:55:05,310 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-23) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 544888fc-39f1-4f2c-8557-8a68e9dfe6cf >2014-10-23 13:55:05,310 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-23) START, SPMClearTaskVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, taskId = 544888fc-39f1-4f2c-8557-8a68e9dfe6cf), log id: 9e3a5a >2014-10-23 13:55:05,312 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-23) START, HSMClearTaskVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, taskId=544888fc-39f1-4f2c-8557-8a68e9dfe6cf), log id: 1408eb07 >2014-10-23 13:55:05,463 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-23) FINISH, HSMClearTaskVDSCommand, log id: 1408eb07 >2014-10-23 13:55:05,464 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-23) FINISH, SPMClearTaskVDSCommand, log id: 9e3a5a >2014-10-23 13:55:05,468 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-23) BaseAsyncTask::removeTaskFromDB: Removed task 544888fc-39f1-4f2c-8557-8a68e9dfe6cf from DataBase >2014-10-23 13:55:05,468 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-23) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 462f7ae2-7812-4357-8a4c-4956391c8b16 >2014-10-23 13:55:42,713 INFO [org.ovirt.engine.core.bll.ChangeVMClusterCommand] (ajp-/127.0.0.1:8702-7) [43760203] Running command: ChangeVMClusterCommand internal: false. Entities affected : ID: 588d43e4-d12e-49b8-b834-734dbcaaf703 Type: VMAction group EDIT_VM_PROPERTIES with role type USER, ID: 67866b36-fd68-4106-8758-34cf31b0c3d4 Type: VdsGroupsAction group CREATE_VM with role type USER >2014-10-23 13:55:42,732 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [43760203] Correlation ID: 43760203, Call Stack: null, Custom Event ID: -1, Message: VM test_memory_vm configuration was updated by admin. >2014-10-23 13:55:42,788 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp-/127.0.0.1:8702-9) [2398852f] Lock Acquired to object EngineLock [exclusiveLocks= key: test_memory_vm value: VM_NAME >, sharedLocks= key: 588d43e4-d12e-49b8-b834-734dbcaaf703 value: VM >] >2014-10-23 13:55:42,841 WARN [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp-/127.0.0.1:8702-9) [2398852f] CanDoAction of action UpdateVm failed. Reasons:VAR__ACTION__UPDATE,VAR__TYPE__VM,ACTION_TYPE_CPU_PROFILE_NOT_MATCH_CLUSTER >2014-10-23 13:55:42,841 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp-/127.0.0.1:8702-9) [2398852f] Lock freed to object EngineLock [exclusiveLocks= key: test_memory_vm value: VM_NAME >, sharedLocks= key: 588d43e4-d12e-49b8-b834-734dbcaaf703 value: VM >] >2014-10-23 13:55:59,751 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp-/127.0.0.1:8702-7) [5a8b35cc] Lock Acquired to object EngineLock [exclusiveLocks= key: test_memory_vm value: VM_NAME >, sharedLocks= key: 588d43e4-d12e-49b8-b834-734dbcaaf703 value: VM >] >2014-10-23 13:55:59,795 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp-/127.0.0.1:8702-7) [5a8b35cc] Running command: UpdateVmCommand internal: false. Entities affected : ID: 588d43e4-d12e-49b8-b834-734dbcaaf703 Type: VMAction group EDIT_VM_PROPERTIES with role type USER >2014-10-23 13:55:59,823 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [5a8b35cc] Correlation ID: 5a8b35cc, Job ID: 60f885b2-1e4e-469f-8b45-9e0aeffd4172, Call Stack: null, Custom Event ID: -1, Message: VM test_memory_vm configuration was updated by admin. >2014-10-23 13:55:59,827 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp-/127.0.0.1:8702-7) [5a8b35cc] Lock freed to object EngineLock [exclusiveLocks= key: test_memory_vm value: VM_NAME >, sharedLocks= key: 588d43e4-d12e-49b8-b834-734dbcaaf703 value: VM >] >2014-10-23 13:56:00,315 INFO [org.ovirt.engine.core.bll.network.vm.AddVmInterfaceCommand] (ajp-/127.0.0.1:8702-3) [91811d1] Running command: AddVmInterfaceCommand internal: false. Entities affected : ID: 588d43e4-d12e-49b8-b834-734dbcaaf703 Type: VMAction group CONFIGURE_VM_NETWORK with role type USER, ID: c078ca43-dc90-4ffe-a17e-68721438ee95 Type: VnicProfileAction group CONFIGURE_VM_NETWORK with role type USER >2014-10-23 13:56:00,383 INFO [org.ovirt.engine.core.bll.network.vm.ActivateDeactivateVmNicCommand] (ajp-/127.0.0.1:8702-3) [2f826121] Running command: ActivateDeactivateVmNicCommand internal: true. Entities affected : ID: 588d43e4-d12e-49b8-b834-734dbcaaf703 Type: VM >2014-10-23 13:56:00,417 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) [2f826121] Correlation ID: 2f826121, Call Stack: null, Custom Event ID: -1, Message: Network Interface nic1 (VirtIO) was plugged to VM test_memory_vm. (User: admin) >2014-10-23 13:56:00,426 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) [2f826121] Correlation ID: 91811d1, Call Stack: null, Custom Event ID: -1, Message: Interface nic1 (VirtIO) was added to VM test_memory_vm. (User: admin) >2014-10-23 13:56:13,208 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp-/127.0.0.1:8702-4) [4fad3e8e] Lock Acquired to object EngineLock [exclusiveLocks= key: 588d43e4-d12e-49b8-b834-734dbcaaf703 value: VM >, sharedLocks= ] >2014-10-23 13:56:13,241 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp-/127.0.0.1:8702-4) [4fad3e8e] START, IsVmDuringInitiatingVDSCommand( vmId = 588d43e4-d12e-49b8-b834-734dbcaaf703), log id: 169d3b4e >2014-10-23 13:56:13,244 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp-/127.0.0.1:8702-4) [4fad3e8e] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 169d3b4e >2014-10-23 13:56:13,257 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ajp-/127.0.0.1:8702-4) [4fad3e8e] Candidate host alma06.qa.lab.tlv.redhat.com (1c39bd88-8672-408f-a593-60d6ef8f647c) was filtered out by VAR__FILTERTYPE__INTERNAL filter Memory >2014-10-23 13:56:13,257 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ajp-/127.0.0.1:8702-4) [4fad3e8e] Candidate host cyan-vdsg.qa.lab.tlv.redhat.com (3a2a45bd-4411-4428-aa45-a514d30d3ea6) was filtered out by VAR__FILTERTYPE__INTERNAL filter Memory >2014-10-23 13:56:13,258 WARN [org.ovirt.engine.core.bll.RunVmCommand] (ajp-/127.0.0.1:8702-4) [4fad3e8e] CanDoAction of action RunVm failed. Reasons:VAR__ACTION__RUN,VAR__TYPE__VM,SCHEDULING_ALL_HOSTS_FILTERED_OUT,VAR__FILTERTYPE__INTERNAL,$hostName cyan-vdsg.qa.lab.tlv.redhat.com,$filterName Memory,VAR__DETAIL__NOT_ENOUGH_MEMORY,SCHEDULING_HOST_FILTERED_REASON_WITH_DETAIL,VAR__FILTERTYPE__INTERNAL,$hostName alma06.qa.lab.tlv.redhat.com,$filterName Memory,VAR__DETAIL__NOT_ENOUGH_MEMORY,SCHEDULING_HOST_FILTERED_REASON_WITH_DETAIL >2014-10-23 13:56:13,258 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp-/127.0.0.1:8702-4) [4fad3e8e] Lock freed to object EngineLock [exclusiveLocks= key: 588d43e4-d12e-49b8-b834-734dbcaaf703 value: VM >, sharedLocks= ] >2014-10-23 13:56:20,091 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-7) Setting new tasks map. The map contains now 0 tasks >2014-10-23 13:56:20,091 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-7) Cleared all tasks of pool f8f5eaee-8fd0-4b45-87db-62d61b03a916. >2014-10-23 13:56:46,447 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp-/127.0.0.1:8702-9) [19dc41ec] Lock Acquired to object EngineLock [exclusiveLocks= key: test_memory_vm value: VM_NAME >, sharedLocks= key: 588d43e4-d12e-49b8-b834-734dbcaaf703 value: VM >] >2014-10-23 13:56:46,482 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp-/127.0.0.1:8702-9) [19dc41ec] Running command: UpdateVmCommand internal: false. Entities affected : ID: 588d43e4-d12e-49b8-b834-734dbcaaf703 Type: VMAction group EDIT_VM_PROPERTIES with role type USER >2014-10-23 13:56:46,504 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [19dc41ec] Correlation ID: 19dc41ec, Job ID: b3f9650f-a4d3-4898-b74b-e0790f3cac3c, Call Stack: null, Custom Event ID: -1, Message: VM test_memory_vm configuration was updated by admin. >2014-10-23 13:56:46,508 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp-/127.0.0.1:8702-9) [19dc41ec] Lock freed to object EngineLock [exclusiveLocks= key: test_memory_vm value: VM_NAME >, sharedLocks= key: 588d43e4-d12e-49b8-b834-734dbcaaf703 value: VM >] >2014-10-23 13:56:49,056 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp-/127.0.0.1:8702-4) [603be14d] Lock Acquired to object EngineLock [exclusiveLocks= key: 588d43e4-d12e-49b8-b834-734dbcaaf703 value: VM >, sharedLocks= ] >2014-10-23 13:56:49,100 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp-/127.0.0.1:8702-4) [603be14d] START, IsVmDuringInitiatingVDSCommand( vmId = 588d43e4-d12e-49b8-b834-734dbcaaf703), log id: 4998ae4e >2014-10-23 13:56:49,100 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp-/127.0.0.1:8702-4) [603be14d] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 4998ae4e >2014-10-23 13:56:49,108 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ajp-/127.0.0.1:8702-4) [603be14d] Candidate host cyan-vdsg.qa.lab.tlv.redhat.com (3a2a45bd-4411-4428-aa45-a514d30d3ea6) was filtered out by VAR__FILTERTYPE__INTERNAL filter Memory >2014-10-23 13:56:49,147 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-7-thread-41) [603be14d] Running command: RunVmCommand internal: false. Entities affected : ID: 588d43e4-d12e-49b8-b834-734dbcaaf703 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-10-23 13:56:49,168 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (org.ovirt.thread.pool-7-thread-41) [603be14d] Candidate host cyan-vdsg.qa.lab.tlv.redhat.com (3a2a45bd-4411-4428-aa45-a514d30d3ea6) was filtered out by VAR__FILTERTYPE__INTERNAL filter Memory (correlation id: 603be14d) >2014-10-23 13:56:49,209 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-7-thread-41) [603be14d] START, UpdateVmDynamicDataVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@89b76df6), log id: 56871fb >2014-10-23 13:56:49,211 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-7-thread-41) [603be14d] FINISH, UpdateVmDynamicDataVDSCommand, log id: 56871fb >2014-10-23 13:56:49,216 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-7-thread-41) [603be14d] START, CreateVmVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vmId=588d43e4-d12e-49b8-b834-734dbcaaf703, vm=VM [test_memory_vm]), log id: 63dc4bf5 >2014-10-23 13:56:49,220 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-7-thread-41) [603be14d] START, CreateVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vmId=588d43e4-d12e-49b8-b834-734dbcaaf703, vm=VM [test_memory_vm]), log id: 63bd70eb >2014-10-23 13:56:49,255 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-7-thread-41) [603be14d] org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand kvmEnable=true,keyboardLayout=en-us,nice=0,displayNetwork=rhevm,copyPasteEnable=true,timeOffset=0,transparentHugePages=true,vmId=588d43e4-d12e-49b8-b834-734dbcaaf703,acpiEnable=true,custom={},spiceSslCipherSuite=DEFAULT,memSize=14000,smp=1,emulatedMachine=rhel6.5.0,vmType=kvm,memGuaranteedSize=14000,display=qxl,smartcardEnable=false,bootMenuEnable=false,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,smpCoresPerSocket=1,maxVCpus=16,devices=[{specParams={vram=32768, heads=1}, device=qxl, type=video, deviceId=5b65bb2d-4bdf-4642-9f90-2ee646949f1d}, {shared=false, iface=ide, index=2, specParams={path=}, path=, device=cdrom, type=disk, readonly=true, deviceId=dcf8c427-fca5-4409-a77e-cf767001b76a}, {shared=false, index=0, volumeID=2457fd51-d920-4394-9ce3-4d801ece1a39, propagateErrors=off, format=raw, type=disk, iface=virtio, bootOrder=1, domainID=a1341057-9cf8-4750-a734-656d05a5d2c5, imageID=59ed3435-ad92-48ff-be5f-c2c1c89b6fa8, specParams={}, optional=false, device=disk, poolID=f8f5eaee-8fd0-4b45-87db-62d61b03a916, readonly=false, deviceId=59ed3435-ad92-48ff-be5f-c2c1c89b6fa8}, {nicModel=pv, specParams={outbound={}, inbound={}}, macAddr=00:1a:4a:91:91:06, device=bridge, linkActive=true, type=interface, filter=vdsm-no-mac-spoofing, network=rhevm, deviceId=653077db-cefc-46e5-87b1-3cd5c0b1093e}, {specParams={model=virtio}, device=memballoon, type=balloon, deviceId=a6f57bf8-9847-402b-833e-3cd769647a96}, {index=0, model=virtio-scsi, specParams={}, device=scsi, type=controller, deviceId=a64badd4-fa3e-40b4-bfdd-02fc3117276e}, {specParams={}, device=virtio-serial, type=controller, deviceId=89f403cf-3d44-43d6-b8d9-a30d52920883}],vmName=test_memory_vm,cpuType=Opteron_G1,fileTransferEnable=true >2014-10-23 13:56:49,316 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-7-thread-41) [603be14d] FINISH, CreateVDSCommand, log id: 63bd70eb >2014-10-23 13:56:49,322 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-7-thread-41) [603be14d] FINISH, CreateVmVDSCommand, return: WaitForLaunch, log id: 63dc4bf5 >2014-10-23 13:56:49,322 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-7-thread-41) [603be14d] Lock freed to object EngineLock [exclusiveLocks= key: 588d43e4-d12e-49b8-b834-734dbcaaf703 value: VM >, sharedLocks= ] >2014-10-23 13:56:49,330 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-41) [603be14d] Correlation ID: 603be14d, Job ID: bee24fa6-0d2b-4e45-b770-75e45ce598ea, Call Stack: null, Custom Event ID: -1, Message: VM test_memory_vm was started by admin (Host: alma06.qa.lab.tlv.redhat.com). >2014-10-23 13:56:49,390 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-23) START, FullListVdsCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c], vmIds=[588d43e4-d12e-49b8-b834-734dbcaaf703]), log id: 6b6b3b59 >2014-10-23 13:56:49,440 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-23) FINISH, FullListVdsCommand, return: [{acpiEnable=true, emulatedMachine=rhel6.5.0, pid=0, memGuaranteedSize=14000, transparentHugePages=true, displaySecurePort=-1, spiceSslCipherSuite=DEFAULT, cpuType=Opteron_G1, smp=1, custom={}, vmType=kvm, memSize=14000, smpCoresPerSocket=1, vmName=test_memory_vm, nice=0, status=WaitForLaunch, bootMenuEnable=false, vmId=588d43e4-d12e-49b8-b834-734dbcaaf703, copyPasteEnable=true, displayIp=10.35.117.30, displayPort=-1, smartcardEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, fileTransferEnable=true, nicModel=rtl8139,pv, keyboardLayout=en-us, kvmEnable=true, displayNetwork=rhevm, devices=[Ljava.lang.Object;@4d058727, timeOffset=0, maxVCpus=16, clientIp=, display=qxl}], log id: 6b6b3b59 >2014-10-23 13:56:49,442 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-23) Received a memballoon Device without an address when processing VM 588d43e4-d12e-49b8-b834-734dbcaaf703 devices, skipping device: {target=14336000, specParams={model=virtio}, deviceType=balloon, deviceId=a6f57bf8-9847-402b-833e-3cd769647a96, device=memballoon, type=balloon} >2014-10-23 13:56:49,442 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-23) Received a scsi Device without an address when processing VM 588d43e4-d12e-49b8-b834-734dbcaaf703 devices, skipping device: {index=0, specParams={}, deviceType=controller, deviceId=a64badd4-fa3e-40b4-bfdd-02fc3117276e, device=scsi, model=virtio-scsi, type=controller} >2014-10-23 13:56:49,442 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-23) Received a virtio-serial Device without an address when processing VM 588d43e4-d12e-49b8-b834-734dbcaaf703 devices, skipping device: {device=virtio-serial, specParams={}, deviceType=controller, type=controller, deviceId=89f403cf-3d44-43d6-b8d9-a30d52920883} >2014-10-23 13:56:49,442 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-23) Received a qxl Device without an address when processing VM 588d43e4-d12e-49b8-b834-734dbcaaf703 devices, skipping device: {device=qxl, specParams={vram=32768, heads=1}, deviceType=video, type=video, deviceId=5b65bb2d-4bdf-4642-9f90-2ee646949f1d} >2014-10-23 13:56:49,442 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-23) Received a spice Device without an address when processing VM 588d43e4-d12e-49b8-b834-734dbcaaf703 devices, skipping device: {device=spice, specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, copyPasteEnable=true, keyMap=en-us, displayIp=10.35.117.30, displayNetwork=rhevm}, deviceType=graphics, type=graphics} >2014-10-23 13:56:49,443 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-23) Received a bridge Device without an address when processing VM 588d43e4-d12e-49b8-b834-734dbcaaf703 devices, skipping device: {nicModel=pv, macAddr=00:1a:4a:91:91:06, linkActive=true, network=rhevm, filter=vdsm-no-mac-spoofing, specParams={inbound={}, outbound={}}, deviceType=interface, deviceId=653077db-cefc-46e5-87b1-3cd5c0b1093e, device=bridge, type=interface} >2014-10-23 13:56:49,443 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-23) Received a cdrom Device without an address when processing VM 588d43e4-d12e-49b8-b834-734dbcaaf703 devices, skipping device: {index=2, iface=ide, specParams={path=}, readonly=true, deviceType=disk, deviceId=dcf8c427-fca5-4409-a77e-cf767001b76a, path=, device=cdrom, shared=false, type=disk} >2014-10-23 13:56:49,443 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-23) Received a disk Device without an address when processing VM 588d43e4-d12e-49b8-b834-734dbcaaf703 devices, skipping device: {volumeInfo={domainID=a1341057-9cf8-4750-a734-656d05a5d2c5, volType=path, leaseOffset=0, path=/rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__storage__rhevm__35/a1341057-9cf8-4750-a734-656d05a5d2c5/images/59ed3435-ad92-48ff-be5f-c2c1c89b6fa8/2457fd51-d920-4394-9ce3-4d801ece1a39, volumeID=2457fd51-d920-4394-9ce3-4d801ece1a39, leasePath=/rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__storage__rhevm__35/a1341057-9cf8-4750-a734-656d05a5d2c5/images/59ed3435-ad92-48ff-be5f-c2c1c89b6fa8/2457fd51-d920-4394-9ce3-4d801ece1a39.lease, imageID=59ed3435-ad92-48ff-be5f-c2c1c89b6fa8}, index=0, iface=virtio, apparentsize=1073741824, imageID=59ed3435-ad92-48ff-be5f-c2c1c89b6fa8, readonly=false, deviceType=disk, shared=false, truesize=24576, type=disk, domainID=a1341057-9cf8-4750-a734-656d05a5d2c5, reqsize=0, format=raw, deviceId=59ed3435-ad92-48ff-be5f-c2c1c89b6fa8, poolID=f8f5eaee-8fd0-4b45-87db-62d61b03a916, device=disk, path=/rhev/data-center/f8f5eaee-8fd0-4b45-87db-62d61b03a916/a1341057-9cf8-4750-a734-656d05a5d2c5/images/59ed3435-ad92-48ff-be5f-c2c1c89b6fa8/2457fd51-d920-4394-9ce3-4d801ece1a39, propagateErrors=off, optional=false, bootOrder=1, volumeID=2457fd51-d920-4394-9ce3-4d801ece1a39, specParams={}, volumeChain=[Ljava.lang.Object;@6d2472c6} >2014-10-23 13:56:52,482 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-82) VM test_memory_vm 588d43e4-d12e-49b8-b834-734dbcaaf703 moved from WaitForLaunch --> PoweringUp >2014-10-23 13:56:52,502 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-82) START, FullListVdsCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c], vmIds=[588d43e4-d12e-49b8-b834-734dbcaaf703]), log id: 41819378 >2014-10-23 13:56:52,517 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-82) FINISH, FullListVdsCommand, return: [{acpiEnable=true, emulatedMachine=rhel6.5.0, pid=28177, memGuaranteedSize=14000, transparentHugePages=true, displaySecurePort=5903, spiceSslCipherSuite=DEFAULT, cpuType=Opteron_G1, smp=1, pauseCode=NOERR, custom={}, vmType=kvm, memSize=14000, smpCoresPerSocket=1, vmName=test_memory_vm, nice=0, status=Up, bootMenuEnable=false, vmId=588d43e4-d12e-49b8-b834-734dbcaaf703, copyPasteEnable=true, displayIp=10.35.117.30, displayPort=5902, smartcardEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, fileTransferEnable=true, nicModel=rtl8139,pv, keyboardLayout=en-us, kvmEnable=true, displayNetwork=rhevm, devices=[Ljava.lang.Object;@5552d933, timeOffset=0, maxVCpus=16, clientIp=, display=qxl}], log id: 41819378 >2014-10-23 13:56:52,519 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-82) Received a spice Device without an address when processing VM 588d43e4-d12e-49b8-b834-734dbcaaf703 devices, skipping device: {tlsPort=5903, specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, copyPasteEnable=true, keyMap=en-us, displayIp=10.35.117.30, displayNetwork=rhevm}, deviceType=graphics, device=spice, type=graphics, port=5902} >2014-10-23 13:57:50,377 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-82) VM test_memory_vm 588d43e4-d12e-49b8-b834-734dbcaaf703 moved from PoweringUp --> Up >2014-10-23 13:57:50,389 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-82) Correlation ID: 603be14d, Job ID: bee24fa6-0d2b-4e45-b770-75e45ce598ea, Call Stack: null, Custom Event ID: -1, Message: VM test_memory_vm started on Host alma06.qa.lab.tlv.redhat.com >2014-10-23 13:58:06,806 INFO [org.ovirt.engine.core.bll.AddVmFromScratchCommand] (ajp-/127.0.0.1:8702-3) [6a353e2] Lock Acquired to object EngineLock [exclusiveLocks= key: test_memory_vm_2 value: VM_NAME >, sharedLocks= ] >2014-10-23 13:58:06,847 INFO [org.ovirt.engine.core.bll.AddVmFromScratchCommand] (ajp-/127.0.0.1:8702-3) [6a353e2] Running command: AddVmFromScratchCommand internal: false. Entities affected : ID: 08ea0169-ae42-4e1b-bb20-839a66cd2010 Type: VdsGroupsAction group CREATE_VM with role type USER >2014-10-23 13:58:06,908 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) [6a353e2] Correlation ID: 6a353e2, Job ID: 3d87b183-eaa2-4a30-a01f-ca6ed99df427, Call Stack: null, Custom Event ID: -1, Message: VM test_memory_vm_2 was created by admin. >2014-10-23 13:58:06,912 INFO [org.ovirt.engine.core.bll.AddVmFromScratchCommand] (ajp-/127.0.0.1:8702-3) [6a353e2] Lock freed to object EngineLock [exclusiveLocks= key: test_memory_vm_2 value: VM_NAME >, sharedLocks= ] >2014-10-23 13:58:07,547 INFO [org.ovirt.engine.core.bll.network.vm.AddVmInterfaceCommand] (ajp-/127.0.0.1:8702-1) [387409bf] Running command: AddVmInterfaceCommand internal: false. Entities affected : ID: 002a1a18-90ff-4420-8d36-700707f95a32 Type: VMAction group CONFIGURE_VM_NETWORK with role type USER, ID: c078ca43-dc90-4ffe-a17e-68721438ee95 Type: VnicProfileAction group CONFIGURE_VM_NETWORK with role type USER >2014-10-23 13:58:07,577 INFO [org.ovirt.engine.core.bll.network.vm.ActivateDeactivateVmNicCommand] (ajp-/127.0.0.1:8702-1) [62d1ee82] Running command: ActivateDeactivateVmNicCommand internal: true. Entities affected : ID: 002a1a18-90ff-4420-8d36-700707f95a32 Type: VM >2014-10-23 13:58:07,630 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [62d1ee82] Correlation ID: 62d1ee82, Call Stack: null, Custom Event ID: -1, Message: Network Interface nic1 (VirtIO) was plugged to VM test_memory_vm_2. (User: admin) >2014-10-23 13:58:07,638 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [62d1ee82] Correlation ID: 387409bf, Call Stack: null, Custom Event ID: -1, Message: Interface nic1 (VirtIO) was added to VM test_memory_vm_2. (User: admin) >2014-10-23 13:58:07,672 INFO [org.ovirt.engine.core.bll.network.vm.ReorderVmNicsCommand] (ajp-/127.0.0.1:8702-4) [bd0fbb6] Running command: ReorderVmNicsCommand internal: false. Entities affected : ID: 002a1a18-90ff-4420-8d36-700707f95a32 Type: VMAction group CREATE_VM with role type USER >2014-10-23 13:58:11,303 INFO [org.ovirt.engine.core.bll.AddDiskCommand] (ajp-/127.0.0.1:8702-4) [349c886b] Lock Acquired to object EngineLock [exclusiveLocks= key: 002a1a18-90ff-4420-8d36-700707f95a32 value: VM_DISK_BOOT >, sharedLocks= key: 002a1a18-90ff-4420-8d36-700707f95a32 value: VM >] >2014-10-23 13:58:11,320 INFO [org.ovirt.engine.core.bll.AddDiskCommand] (ajp-/127.0.0.1:8702-4) [349c886b] Running command: AddDiskCommand internal: false. Entities affected : ID: 002a1a18-90ff-4420-8d36-700707f95a32 Type: VMAction group CONFIGURE_VM_STORAGE with role type USER, ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: StorageAction group CREATE_DISK with role type USER >2014-10-23 13:58:11,436 INFO [org.ovirt.engine.core.bll.AddImageFromScratchCommand] (ajp-/127.0.0.1:8702-4) [3067fa70] Running command: AddImageFromScratchCommand internal: true. Entities affected : ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: Storage >2014-10-23 13:58:11,444 INFO [org.ovirt.engine.core.bll.AddImageFromScratchCommand] (ajp-/127.0.0.1:8702-4) [3067fa70] Lock freed to object EngineLock [exclusiveLocks= key: 002a1a18-90ff-4420-8d36-700707f95a32 value: VM_DISK_BOOT >, sharedLocks= key: 002a1a18-90ff-4420-8d36-700707f95a32 value: VM >] >2014-10-23 13:58:11,444 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (ajp-/127.0.0.1:8702-4) [3067fa70] START, CreateImageVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = 903cc803-e322-467d-ab3d-a538e86b48a2, imageSizeInBytes = 1073741824, volumeFormat = RAW, newImageId = 164fef3b-77fd-4f6b-9f2d-f51a31a256d0, newImageDescription = ), log id: 4006dbc9 >2014-10-23 13:58:11,445 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (ajp-/127.0.0.1:8702-4) [3067fa70] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-10-23 13:58:11,696 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (ajp-/127.0.0.1:8702-4) [3067fa70] FINISH, CreateImageVDSCommand, return: 164fef3b-77fd-4f6b-9f2d-f51a31a256d0, log id: 4006dbc9 >2014-10-23 13:58:11,702 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (ajp-/127.0.0.1:8702-4) [3067fa70] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command dc71f476-281e-4cb7-8b15-2488fd540fc2 >2014-10-23 13:58:11,702 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (ajp-/127.0.0.1:8702-4) [3067fa70] CommandMultiAsyncTasks::AttachTask: Attaching task ee458792-45b1-4b91-9c87-43d1321690c2 to command dc71f476-281e-4cb7-8b15-2488fd540fc2. >2014-10-23 13:58:11,746 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (ajp-/127.0.0.1:8702-4) [3067fa70] Adding task ee458792-45b1-4b91-9c87-43d1321690c2 (Parent Command AddDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-10-23 13:58:11,797 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [3067fa70] Correlation ID: 349c886b, Job ID: d8f69c4e-62e2-41df-9109-fe8d5efec618, Call Stack: null, Custom Event ID: -1, Message: Add-Disk operation of test_memory_vm_2_Disk1 was initiated on VM test_memory_vm_2 by admin. >2014-10-23 13:58:11,797 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (ajp-/127.0.0.1:8702-4) [3067fa70] BaseAsyncTask::startPollingTask: Starting to poll task ee458792-45b1-4b91-9c87-43d1321690c2. >2014-10-23 13:58:15,211 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-7) Polling and updating Async Tasks: 1 tasks, 1 tasks to poll now >2014-10-23 13:58:15,221 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-7) SPMAsyncTask::PollTask: Polling task ee458792-45b1-4b91-9c87-43d1321690c2 (Parent Command AddDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-10-23 13:58:15,227 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-7) BaseAsyncTask::onTaskEndSuccess: Task ee458792-45b1-4b91-9c87-43d1321690c2 (Parent Command AddDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-10-23 13:58:15,231 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-7) CommandAsyncTask::endActionIfNecessary: All tasks of command dc71f476-281e-4cb7-8b15-2488fd540fc2 has ended -> executing endAction >2014-10-23 13:58:15,231 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-7) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: dc71f476-281e-4cb7-8b15-2488fd540fc2): calling endAction . >2014-10-23 13:58:15,231 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-48) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction AddDisk, executionIndex: 0 >2014-10-23 13:58:15,260 INFO [org.ovirt.engine.core.bll.AddDiskCommand] (org.ovirt.thread.pool-7-thread-48) [349c886b] Ending command successfully: org.ovirt.engine.core.bll.AddDiskCommand >2014-10-23 13:58:15,276 INFO [org.ovirt.engine.core.bll.AddImageFromScratchCommand] (org.ovirt.thread.pool-7-thread-48) [3067fa70] Ending command successfully: org.ovirt.engine.core.bll.AddImageFromScratchCommand >2014-10-23 13:58:15,290 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-48) [3067fa70] START, GetImageInfoVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = 903cc803-e322-467d-ab3d-a538e86b48a2, imageId = 164fef3b-77fd-4f6b-9f2d-f51a31a256d0), log id: 49a3a906 >2014-10-23 13:58:15,327 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-7-thread-48) [3067fa70] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@8e9eab33, log id: 49a3a906 >2014-10-23 13:58:15,343 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-48) Correlation ID: 349c886b, Job ID: d8f69c4e-62e2-41df-9109-fe8d5efec618, Call Stack: null, Custom Event ID: -1, Message: The disk test_memory_vm_2_Disk1 was successfully added to VM test_memory_vm_2. >2014-10-23 13:58:15,344 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-48) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddDisk completed, handling the result. >2014-10-23 13:58:15,344 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-48) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddDisk succeeded, clearing tasks. >2014-10-23 13:58:15,351 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-48) SPMAsyncTask::ClearAsyncTask: Attempting to clear task ee458792-45b1-4b91-9c87-43d1321690c2 >2014-10-23 13:58:15,351 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-48) START, SPMClearTaskVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, taskId = ee458792-45b1-4b91-9c87-43d1321690c2), log id: 5f83251c >2014-10-23 13:58:15,352 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-48) START, HSMClearTaskVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, taskId=ee458792-45b1-4b91-9c87-43d1321690c2), log id: 1af98241 >2014-10-23 13:58:15,397 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-48) FINISH, HSMClearTaskVDSCommand, log id: 1af98241 >2014-10-23 13:58:15,397 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-48) FINISH, SPMClearTaskVDSCommand, log id: 5f83251c >2014-10-23 13:58:15,401 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-48) BaseAsyncTask::removeTaskFromDB: Removed task ee458792-45b1-4b91-9c87-43d1321690c2 from DataBase >2014-10-23 13:58:15,401 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-48) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity dc71f476-281e-4cb7-8b15-2488fd540fc2 >2014-10-23 13:58:17,845 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp-/127.0.0.1:8702-5) [49b6cd87] Lock Acquired to object EngineLock [exclusiveLocks= key: 002a1a18-90ff-4420-8d36-700707f95a32 value: VM >, sharedLocks= ] >2014-10-23 13:58:17,887 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp-/127.0.0.1:8702-5) [49b6cd87] START, IsVmDuringInitiatingVDSCommand( vmId = 002a1a18-90ff-4420-8d36-700707f95a32), log id: 54746eab >2014-10-23 13:58:17,888 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp-/127.0.0.1:8702-5) [49b6cd87] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 54746eab >2014-10-23 13:58:17,894 WARN [org.ovirt.engine.core.bll.RunVmCommand] (ajp-/127.0.0.1:8702-5) [49b6cd87] CanDoAction of action RunVm failed. Reasons:VAR__ACTION__RUN,VAR__TYPE__VM,SCHEDULING_NO_HOSTS >2014-10-23 13:58:17,894 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp-/127.0.0.1:8702-5) [49b6cd87] Lock freed to object EngineLock [exclusiveLocks= key: 002a1a18-90ff-4420-8d36-700707f95a32 value: VM >, sharedLocks= ] >2014-10-23 13:58:25,383 INFO [org.ovirt.engine.core.bll.ChangeVMClusterCommand] (ajp-/127.0.0.1:8702-5) [7790519f] Running command: ChangeVMClusterCommand internal: false. Entities affected : ID: 002a1a18-90ff-4420-8d36-700707f95a32 Type: VMAction group EDIT_VM_PROPERTIES with role type USER, ID: 67866b36-fd68-4106-8758-34cf31b0c3d4 Type: VdsGroupsAction group CREATE_VM with role type USER >2014-10-23 13:58:25,399 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [7790519f] Correlation ID: 7790519f, Call Stack: null, Custom Event ID: -1, Message: VM test_memory_vm_2 configuration was updated by admin. >2014-10-23 13:58:25,446 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp-/127.0.0.1:8702-2) [6cc66552] Lock Acquired to object EngineLock [exclusiveLocks= key: test_memory_vm_2 value: VM_NAME >, sharedLocks= key: 002a1a18-90ff-4420-8d36-700707f95a32 value: VM >] >2014-10-23 13:58:25,543 WARN [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp-/127.0.0.1:8702-2) [6cc66552] CanDoAction of action UpdateVm failed. Reasons:VAR__ACTION__UPDATE,VAR__TYPE__VM,ACTION_TYPE_CPU_PROFILE_NOT_MATCH_CLUSTER >2014-10-23 13:58:25,543 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp-/127.0.0.1:8702-2) [6cc66552] Lock freed to object EngineLock [exclusiveLocks= key: test_memory_vm_2 value: VM_NAME >, sharedLocks= key: 002a1a18-90ff-4420-8d36-700707f95a32 value: VM >] >2014-10-23 13:58:46,039 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp-/127.0.0.1:8702-8) [202e046a] Lock Acquired to object EngineLock [exclusiveLocks= key: test_memory_vm_2 value: VM_NAME >, sharedLocks= key: 002a1a18-90ff-4420-8d36-700707f95a32 value: VM >] >2014-10-23 13:58:46,076 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp-/127.0.0.1:8702-8) [202e046a] Running command: UpdateVmCommand internal: false. Entities affected : ID: 002a1a18-90ff-4420-8d36-700707f95a32 Type: VMAction group EDIT_VM_PROPERTIES with role type USER >2014-10-23 13:58:46,117 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-8) [202e046a] Correlation ID: 202e046a, Job ID: b4099909-cd8b-46c5-a6a5-a171a2aeafda, Call Stack: null, Custom Event ID: -1, Message: VM test_memory_vm_2 configuration was updated by admin. >2014-10-23 13:58:46,122 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp-/127.0.0.1:8702-8) [202e046a] Lock freed to object EngineLock [exclusiveLocks= key: test_memory_vm_2 value: VM_NAME >, sharedLocks= key: 002a1a18-90ff-4420-8d36-700707f95a32 value: VM >] >2014-10-23 13:58:48,567 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp-/127.0.0.1:8702-5) [47e0f5a8] Lock Acquired to object EngineLock [exclusiveLocks= key: 002a1a18-90ff-4420-8d36-700707f95a32 value: VM >, sharedLocks= ] >2014-10-23 13:58:48,602 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp-/127.0.0.1:8702-5) [47e0f5a8] START, IsVmDuringInitiatingVDSCommand( vmId = 002a1a18-90ff-4420-8d36-700707f95a32), log id: 770ffb14 >2014-10-23 13:58:48,602 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp-/127.0.0.1:8702-5) [47e0f5a8] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 770ffb14 >2014-10-23 13:58:48,614 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ajp-/127.0.0.1:8702-5) [47e0f5a8] Candidate host cyan-vdsg.qa.lab.tlv.redhat.com (3a2a45bd-4411-4428-aa45-a514d30d3ea6) was filtered out by VAR__FILTERTYPE__INTERNAL filter Memory >2014-10-23 13:58:48,694 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-7-thread-39) [47e0f5a8] Running command: RunVmCommand internal: false. Entities affected : ID: 002a1a18-90ff-4420-8d36-700707f95a32 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-10-23 13:58:48,713 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (org.ovirt.thread.pool-7-thread-39) [47e0f5a8] Candidate host cyan-vdsg.qa.lab.tlv.redhat.com (3a2a45bd-4411-4428-aa45-a514d30d3ea6) was filtered out by VAR__FILTERTYPE__INTERNAL filter Memory (correlation id: 47e0f5a8) >2014-10-23 13:58:48,748 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-7-thread-39) [47e0f5a8] START, UpdateVmDynamicDataVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@f3083820), log id: 100cd630 >2014-10-23 13:58:48,782 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-7-thread-39) [47e0f5a8] FINISH, UpdateVmDynamicDataVDSCommand, log id: 100cd630 >2014-10-23 13:58:48,787 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-7-thread-39) [47e0f5a8] START, CreateVmVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vmId=002a1a18-90ff-4420-8d36-700707f95a32, vm=VM [test_memory_vm_2]), log id: 69d906ac >2014-10-23 13:58:48,810 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-7-thread-39) [47e0f5a8] START, CreateVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vmId=002a1a18-90ff-4420-8d36-700707f95a32, vm=VM [test_memory_vm_2]), log id: 5c31da2a >2014-10-23 13:58:48,879 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-7-thread-39) [47e0f5a8] org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand kvmEnable=true,keyboardLayout=en-us,nice=0,displayNetwork=rhevm,copyPasteEnable=true,timeOffset=0,transparentHugePages=true,vmId=002a1a18-90ff-4420-8d36-700707f95a32,acpiEnable=true,custom={},spiceSslCipherSuite=DEFAULT,memSize=14000,smp=1,emulatedMachine=rhel6.5.0,vmType=kvm,memGuaranteedSize=14000,display=qxl,smartcardEnable=false,bootMenuEnable=false,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,smpCoresPerSocket=1,maxVCpus=16,devices=[{specParams={vram=32768, heads=1}, device=qxl, type=video, deviceId=761f181e-1e72-4a62-9a9b-6e8ec94b2a98}, {shared=false, iface=ide, index=2, specParams={path=}, path=, device=cdrom, type=disk, readonly=true, deviceId=ace8af45-44cc-4ee4-b7d2-17ea0e615488}, {shared=false, index=0, volumeID=164fef3b-77fd-4f6b-9f2d-f51a31a256d0, propagateErrors=off, format=raw, type=disk, iface=virtio, bootOrder=1, domainID=a1341057-9cf8-4750-a734-656d05a5d2c5, imageID=903cc803-e322-467d-ab3d-a538e86b48a2, specParams={}, optional=false, device=disk, poolID=f8f5eaee-8fd0-4b45-87db-62d61b03a916, readonly=false, deviceId=903cc803-e322-467d-ab3d-a538e86b48a2}, {nicModel=pv, specParams={outbound={}, inbound={}}, macAddr=00:1a:4a:91:91:07, device=bridge, linkActive=true, type=interface, filter=vdsm-no-mac-spoofing, network=rhevm, deviceId=70d45290-51a1-4c9f-8975-56943c4791a5}, {specParams={model=virtio}, device=memballoon, type=balloon, deviceId=d42fd513-65c5-4c15-8998-110268c0e7dc}, {index=0, model=virtio-scsi, specParams={}, device=scsi, type=controller, deviceId=9346ed8f-aa63-488f-8e11-1c2c20777bee}, {specParams={}, device=virtio-serial, type=controller, deviceId=d251afb7-1f42-4b8f-adf5-832b15a0c3d9}],vmName=test_memory_vm_2,cpuType=Opteron_G1,fileTransferEnable=true >2014-10-23 13:58:48,936 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-7-thread-39) [47e0f5a8] FINISH, CreateVDSCommand, log id: 5c31da2a >2014-10-23 13:58:48,942 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-7-thread-39) [47e0f5a8] FINISH, CreateVmVDSCommand, return: WaitForLaunch, log id: 69d906ac >2014-10-23 13:58:48,943 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-7-thread-39) [47e0f5a8] Lock freed to object EngineLock [exclusiveLocks= key: 002a1a18-90ff-4420-8d36-700707f95a32 value: VM >, sharedLocks= ] >2014-10-23 13:58:48,947 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-39) [47e0f5a8] Correlation ID: 47e0f5a8, Job ID: a5bbd575-d4b8-4320-b435-a8283ad88850, Call Stack: null, Custom Event ID: -1, Message: VM test_memory_vm_2 was started by admin (Host: alma06.qa.lab.tlv.redhat.com). >2014-10-23 13:58:51,506 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-5) VM test_memory_vm_2 002a1a18-90ff-4420-8d36-700707f95a32 moved from WaitForLaunch --> PoweringUp >2014-10-23 13:58:51,506 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-5) START, FullListVdsCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c], vmIds=[002a1a18-90ff-4420-8d36-700707f95a32]), log id: 4b15753b >2014-10-23 13:58:51,521 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-5) FINISH, FullListVdsCommand, return: [{acpiEnable=true, emulatedMachine=rhel6.5.0, pid=28547, memGuaranteedSize=14000, transparentHugePages=true, displaySecurePort=5905, spiceSslCipherSuite=DEFAULT, cpuType=Opteron_G1, smp=1, pauseCode=NOERR, custom={}, vmType=kvm, memSize=14000, smpCoresPerSocket=1, vmName=test_memory_vm_2, nice=0, status=Up, bootMenuEnable=false, vmId=002a1a18-90ff-4420-8d36-700707f95a32, copyPasteEnable=true, displayIp=10.35.117.30, displayPort=5904, smartcardEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, fileTransferEnable=true, nicModel=rtl8139,pv, keyboardLayout=en-us, kvmEnable=true, displayNetwork=rhevm, devices=[Ljava.lang.Object;@6dc39aff, timeOffset=0, maxVCpus=16, clientIp=, display=qxl}], log id: 4b15753b >2014-10-23 13:58:51,523 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-5) Received a spice Device without an address when processing VM 002a1a18-90ff-4420-8d36-700707f95a32 devices, skipping device: {tlsPort=5905, specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, copyPasteEnable=true, keyMap=en-us, displayIp=10.35.117.30, displayNetwork=rhevm}, deviceType=graphics, device=spice, type=graphics, port=5904} >2014-10-23 13:59:20,107 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-25) Setting new tasks map. The map contains now 0 tasks >2014-10-23 13:59:20,108 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-25) Cleared all tasks of pool f8f5eaee-8fd0-4b45-87db-62d61b03a916. >2014-10-23 13:59:49,732 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-24) VM test_memory_vm_2 002a1a18-90ff-4420-8d36-700707f95a32 moved from PoweringUp --> Up >2014-10-23 13:59:49,745 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-24) Correlation ID: 47e0f5a8, Job ID: a5bbd575-d4b8-4320-b435-a8283ad88850, Call Stack: null, Custom Event ID: -1, Message: VM test_memory_vm_2 started on Host alma06.qa.lab.tlv.redhat.com >2014-10-23 14:05:27,716 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (ajp-/127.0.0.1:8702-6) [6eb5a038] Lock Acquired to object EngineLock [exclusiveLocks= key: 1c39bd88-8672-408f-a593-60d6ef8f647c value: VDS >, sharedLocks= ] >2014-10-23 14:05:27,718 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (org.ovirt.thread.pool-7-thread-30) [6eb5a038] Running command: RefreshHostCapabilitiesCommand internal: false. Entities affected : ID: 1c39bd88-8672-408f-a593-60d6ef8f647c Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-10-23 14:05:27,720 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (org.ovirt.thread.pool-7-thread-30) [6eb5a038] Before acquiring lock in order to prevent monitoring for host alma06.qa.lab.tlv.redhat.com from data-center dc_35 >2014-10-23 14:05:27,720 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (org.ovirt.thread.pool-7-thread-30) [6eb5a038] Lock acquired, from now a monitoring of host will be skipped for host alma06.qa.lab.tlv.redhat.com from data-center dc_35 >2014-10-23 14:05:27,788 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (org.ovirt.thread.pool-7-thread-30) [6eb5a038] START, GetHardwareInfoVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c]), log id: 23903499 >2014-10-23 14:05:27,796 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (org.ovirt.thread.pool-7-thread-30) [6eb5a038] FINISH, GetHardwareInfoVDSCommand, log id: 23903499 >2014-10-23 14:05:27,833 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (org.ovirt.thread.pool-7-thread-30) [5ac6c245] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 1c39bd88-8672-408f-a593-60d6ef8f647c Type: VDS >2014-10-23 14:05:27,838 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (org.ovirt.thread.pool-7-thread-30) [44de9d13] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 1c39bd88-8672-408f-a593-60d6ef8f647c Type: VDS >2014-10-23 14:05:27,840 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (org.ovirt.thread.pool-7-thread-30) [44de9d13] Refresh host capabilities finished. Lock released. Monitoring can run now for host alma06.qa.lab.tlv.redhat.com from data-center dc_35 >2014-10-23 14:05:27,867 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-30) [44de9d13] Correlation ID: 6eb5a038, Call Stack: null, Custom Event ID: -1, Message: Successfully refreshed the capabilities of host alma06.qa.lab.tlv.redhat.com. >2014-10-23 14:05:27,867 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (org.ovirt.thread.pool-7-thread-30) [44de9d13] Lock freed to object EngineLock [exclusiveLocks= key: 1c39bd88-8672-408f-a593-60d6ef8f647c value: VDS >, sharedLocks= ] >2014-10-23 14:05:28,887 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (ajp-/127.0.0.1:8702-7) [4f0687e5] Lock Acquired to object EngineLock [exclusiveLocks= key: 1c39bd88-8672-408f-a593-60d6ef8f647c value: VDS >, sharedLocks= ] >2014-10-23 14:05:28,890 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (org.ovirt.thread.pool-7-thread-42) [4f0687e5] Running command: RefreshHostCapabilitiesCommand internal: false. Entities affected : ID: 1c39bd88-8672-408f-a593-60d6ef8f647c Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-10-23 14:05:28,890 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (org.ovirt.thread.pool-7-thread-42) [4f0687e5] Before acquiring lock in order to prevent monitoring for host alma06.qa.lab.tlv.redhat.com from data-center dc_35 >2014-10-23 14:05:28,890 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (org.ovirt.thread.pool-7-thread-42) [4f0687e5] Lock acquired, from now a monitoring of host will be skipped for host alma06.qa.lab.tlv.redhat.com from data-center dc_35 >2014-10-23 14:05:28,955 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (org.ovirt.thread.pool-7-thread-42) [4f0687e5] START, GetHardwareInfoVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c]), log id: 549d549 >2014-10-23 14:05:28,967 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (org.ovirt.thread.pool-7-thread-42) [4f0687e5] FINISH, GetHardwareInfoVDSCommand, log id: 549d549 >2014-10-23 14:05:29,073 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (org.ovirt.thread.pool-7-thread-42) [5a0f2f5] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 1c39bd88-8672-408f-a593-60d6ef8f647c Type: VDS >2014-10-23 14:05:29,087 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (org.ovirt.thread.pool-7-thread-42) [6536e1b5] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 1c39bd88-8672-408f-a593-60d6ef8f647c Type: VDS >2014-10-23 14:05:29,100 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (org.ovirt.thread.pool-7-thread-42) [6536e1b5] Refresh host capabilities finished. Lock released. Monitoring can run now for host alma06.qa.lab.tlv.redhat.com from data-center dc_35 >2014-10-23 14:05:29,131 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-42) [6536e1b5] Correlation ID: 4f0687e5, Call Stack: null, Custom Event ID: -1, Message: Successfully refreshed the capabilities of host alma06.qa.lab.tlv.redhat.com. >2014-10-23 14:05:29,131 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (org.ovirt.thread.pool-7-thread-42) [6536e1b5] Lock freed to object EngineLock [exclusiveLocks= key: 1c39bd88-8672-408f-a593-60d6ef8f647c value: VDS >, sharedLocks= ] >2014-10-23 14:05:29,721 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (ajp-/127.0.0.1:8702-10) [6366a8a4] Lock Acquired to object EngineLock [exclusiveLocks= key: 1c39bd88-8672-408f-a593-60d6ef8f647c value: VDS >, sharedLocks= ] >2014-10-23 14:05:29,724 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (org.ovirt.thread.pool-7-thread-16) [6366a8a4] Running command: RefreshHostCapabilitiesCommand internal: false. Entities affected : ID: 1c39bd88-8672-408f-a593-60d6ef8f647c Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-10-23 14:05:29,724 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (org.ovirt.thread.pool-7-thread-16) [6366a8a4] Before acquiring lock in order to prevent monitoring for host alma06.qa.lab.tlv.redhat.com from data-center dc_35 >2014-10-23 14:05:29,724 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (org.ovirt.thread.pool-7-thread-16) [6366a8a4] Lock acquired, from now a monitoring of host will be skipped for host alma06.qa.lab.tlv.redhat.com from data-center dc_35 >2014-10-23 14:05:29,778 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (org.ovirt.thread.pool-7-thread-16) [6366a8a4] START, GetHardwareInfoVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c]), log id: 190de2e9 >2014-10-23 14:05:29,785 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (org.ovirt.thread.pool-7-thread-16) [6366a8a4] FINISH, GetHardwareInfoVDSCommand, log id: 190de2e9 >2014-10-23 14:05:29,820 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (org.ovirt.thread.pool-7-thread-16) [5347a314] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 1c39bd88-8672-408f-a593-60d6ef8f647c Type: VDS >2014-10-23 14:05:29,835 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (org.ovirt.thread.pool-7-thread-16) [7304d3e5] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 1c39bd88-8672-408f-a593-60d6ef8f647c Type: VDS >2014-10-23 14:05:29,837 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (org.ovirt.thread.pool-7-thread-16) [7304d3e5] Refresh host capabilities finished. Lock released. Monitoring can run now for host alma06.qa.lab.tlv.redhat.com from data-center dc_35 >2014-10-23 14:05:29,851 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-16) [7304d3e5] Correlation ID: 6366a8a4, Call Stack: null, Custom Event ID: -1, Message: Successfully refreshed the capabilities of host alma06.qa.lab.tlv.redhat.com. >2014-10-23 14:05:29,851 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (org.ovirt.thread.pool-7-thread-16) [7304d3e5] Lock freed to object EngineLock [exclusiveLocks= key: 1c39bd88-8672-408f-a593-60d6ef8f647c value: VDS >, sharedLocks= ] >2014-10-23 14:05:33,081 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (ajp-/127.0.0.1:8702-7) [1b4ea867] Lock Acquired to object EngineLock [exclusiveLocks= key: 1c39bd88-8672-408f-a593-60d6ef8f647c value: VDS >, sharedLocks= ] >2014-10-23 14:05:33,084 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (org.ovirt.thread.pool-7-thread-27) [1b4ea867] Running command: RefreshHostCapabilitiesCommand internal: false. Entities affected : ID: 1c39bd88-8672-408f-a593-60d6ef8f647c Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-10-23 14:05:33,084 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (org.ovirt.thread.pool-7-thread-27) [1b4ea867] Before acquiring lock in order to prevent monitoring for host alma06.qa.lab.tlv.redhat.com from data-center dc_35 >2014-10-23 14:05:33,085 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (org.ovirt.thread.pool-7-thread-27) [1b4ea867] Lock acquired, from now a monitoring of host will be skipped for host alma06.qa.lab.tlv.redhat.com from data-center dc_35 >2014-10-23 14:05:33,139 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (org.ovirt.thread.pool-7-thread-27) [1b4ea867] START, GetHardwareInfoVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c]), log id: 2653e6b7 >2014-10-23 14:05:33,146 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (org.ovirt.thread.pool-7-thread-27) [1b4ea867] FINISH, GetHardwareInfoVDSCommand, log id: 2653e6b7 >2014-10-23 14:05:33,174 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (org.ovirt.thread.pool-7-thread-27) [3f15430] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 1c39bd88-8672-408f-a593-60d6ef8f647c Type: VDS >2014-10-23 14:05:33,179 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (org.ovirt.thread.pool-7-thread-27) [75712445] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 1c39bd88-8672-408f-a593-60d6ef8f647c Type: VDS >2014-10-23 14:05:33,180 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (org.ovirt.thread.pool-7-thread-27) [75712445] Refresh host capabilities finished. Lock released. Monitoring can run now for host alma06.qa.lab.tlv.redhat.com from data-center dc_35 >2014-10-23 14:05:33,189 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-27) [75712445] Correlation ID: 1b4ea867, Call Stack: null, Custom Event ID: -1, Message: Successfully refreshed the capabilities of host alma06.qa.lab.tlv.redhat.com. >2014-10-23 14:05:33,189 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (org.ovirt.thread.pool-7-thread-27) [75712445] Lock freed to object EngineLock [exclusiveLocks= key: 1c39bd88-8672-408f-a593-60d6ef8f647c value: VDS >, sharedLocks= ] >2014-10-23 14:05:37,356 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (ajp-/127.0.0.1:8702-6) [420a3404] Lock Acquired to object EngineLock [exclusiveLocks= key: 1c39bd88-8672-408f-a593-60d6ef8f647c value: VDS >, sharedLocks= ] >2014-10-23 14:05:37,360 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (org.ovirt.thread.pool-7-thread-3) [420a3404] Running command: RefreshHostCapabilitiesCommand internal: false. Entities affected : ID: 1c39bd88-8672-408f-a593-60d6ef8f647c Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-10-23 14:05:37,360 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (org.ovirt.thread.pool-7-thread-3) [420a3404] Before acquiring lock in order to prevent monitoring for host alma06.qa.lab.tlv.redhat.com from data-center dc_35 >2014-10-23 14:05:37,360 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (org.ovirt.thread.pool-7-thread-3) [420a3404] Lock acquired, from now a monitoring of host will be skipped for host alma06.qa.lab.tlv.redhat.com from data-center dc_35 >2014-10-23 14:05:37,439 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (org.ovirt.thread.pool-7-thread-3) [420a3404] START, GetHardwareInfoVDSCommand(HostName = alma06.qa.lab.tlv.redhat.com, HostId = 1c39bd88-8672-408f-a593-60d6ef8f647c, vds=Host[alma06.qa.lab.tlv.redhat.com,1c39bd88-8672-408f-a593-60d6ef8f647c]), log id: 163ebef5 >2014-10-23 14:05:37,446 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (org.ovirt.thread.pool-7-thread-3) [420a3404] FINISH, GetHardwareInfoVDSCommand, log id: 163ebef5 >2014-10-23 14:05:37,480 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (org.ovirt.thread.pool-7-thread-3) [628fbb9e] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 1c39bd88-8672-408f-a593-60d6ef8f647c Type: VDS >2014-10-23 14:05:37,485 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (org.ovirt.thread.pool-7-thread-3) [4610f5b6] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 1c39bd88-8672-408f-a593-60d6ef8f647c Type: VDS >2014-10-23 14:05:37,487 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (org.ovirt.thread.pool-7-thread-3) [4610f5b6] Refresh host capabilities finished. Lock released. Monitoring can run now for host alma06.qa.lab.tlv.redhat.com from data-center dc_35 >2014-10-23 14:05:37,491 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-3) [4610f5b6] Correlation ID: 420a3404, Call Stack: null, Custom Event ID: -1, Message: Successfully refreshed the capabilities of host alma06.qa.lab.tlv.redhat.com. >2014-10-23 14:05:37,491 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (org.ovirt.thread.pool-7-thread-3) [4610f5b6] Lock freed to object EngineLock [exclusiveLocks= key: 1c39bd88-8672-408f-a593-60d6ef8f647c value: VDS >, sharedLocks= ] >2014-10-23 14:14:12,263 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-4) Running command: LoginUserCommand internal: false. >2014-10-23 14:14:12,269 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-10-23 14:14:12,474 INFO [org.ovirt.engine.core.bll.aaa.LogoutBySessionCommand] (ajp-/127.0.0.1:8702-4) [33b620c] Running command: LogoutBySessionCommand internal: false. >2014-10-23 14:14:12,478 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-4) [50db0926] Running command: LogoutUserCommand internal: false. >2014-10-23 14:14:12,482 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [50db0926] Correlation ID: 50db0926, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-10-23 14:14:15,764 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-1) Running command: LoginUserCommand internal: false. >2014-10-23 14:14:16,329 INFO [org.ovirt.engine.core.bll.aaa.LogoutBySessionCommand] (ajp-/127.0.0.1:8702-1) [39a6ef94] Running command: LogoutBySessionCommand internal: false. >2014-10-23 14:14:16,334 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-1) [5a323502] Running command: LogoutUserCommand internal: false. >2014-10-23 14:14:16,337 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [5a323502] Correlation ID: 5a323502, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-10-23 14:14:35,012 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-5) Running command: LoginUserCommand internal: false. >2014-10-23 14:14:35,015 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-10-23 14:14:35,600 INFO [org.ovirt.engine.core.bll.aaa.LogoutBySessionCommand] (ajp-/127.0.0.1:8702-5) [25a0e7eb] Running command: LogoutBySessionCommand internal: false. >2014-10-23 14:14:35,603 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-5) [bc4356e] Running command: LogoutUserCommand internal: false. >2014-10-23 14:14:35,606 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [bc4356e] Correlation ID: bc4356e, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-10-23 14:15:04,627 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-4) Running command: LoginUserCommand internal: false. >2014-10-23 14:15:04,630 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-10-23 14:32:19,283 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-99) Principal admin::internal synchronized >2014-10-23 14:32:24,917 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-97) Attempting to update VMs/Templates Ovf. >2014-10-23 14:32:25,139 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-97) [53492f05] Lock Acquired to object EngineLock [exclusiveLocks= key: 1742b7af-bb81-4e6b-bf10-feab68a2f799 value: DISK >key: a1341057-9cf8-4750-a734-656d05a5d2c5 value: STORAGE >key: dfba228f-46f3-4758-a4f0-05857683a088 value: DISK >, sharedLocks= key: f8f5eaee-8fd0-4b45-87db-62d61b03a916 value: OVF_UPDATE >] >2014-10-23 14:32:25,143 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-97) [53492f05] Running command: ProcessOvfUpdateForStorageDomainCommand internal: true. Entities affected : ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: Storage >2014-10-23 14:32:25,164 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-97) [53492f05] START, SetVolumeDescriptionVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = dfba228f-46f3-4758-a4f0-05857683a088, imageId = 26155adf-9478-4927-8c49-8e12088c8648), log id: 1d5a3aea >2014-10-23 14:32:25,164 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-97) [53492f05] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-10-23 14:32:25,164 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-97) [53492f05] -- setVolumeDescription parameters: > spUUID=f8f5eaee-8fd0-4b45-87db-62d61b03a916 > sdUUID=a1341057-9cf8-4750-a734-656d05a5d2c5 > imageGroupGUID=dfba228f-46f3-4758-a4f0-05857683a088 > volUUID=26155adf-9478-4927-8c49-8e12088c8648 > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"a1341057-9cf8-4750-a734-656d05a5d2c5"}],"Last Updated":"Wed Oct 22 17:32:20 IDT 2014"} > >2014-10-23 14:32:25,245 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-97) [53492f05] FINISH, SetVolumeDescriptionVDSCommand, log id: 1d5a3aea >2014-10-23 14:32:25,285 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-97) [74c3027a] Running command: UploadStreamCommand internal: true. Entities affected : ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: Storage >2014-10-23 14:32:25,287 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-97) [74c3027a] START, UploadStreamVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6), log id: 49313047 >2014-10-23 14:32:25,287 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-97) [74c3027a] -- executeVdsBrokerCommand: >2014-10-23 14:32:25,287 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-97) [74c3027a] -- parameters: > spUUID=f8f5eaee-8fd0-4b45-87db-62d61b03a916 > sdUUID=a1341057-9cf8-4750-a734-656d05a5d2c5 > imageGUID=dfba228f-46f3-4758-a4f0-05857683a088 > volUUID=26155adf-9478-4927-8c49-8e12088c8648 > size=71,680 > >2014-10-23 14:32:27,373 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-97) [74c3027a] FINISH, UploadStreamVDSCommand, log id: 49313047 >2014-10-23 14:32:27,374 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-97) [74c3027a] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 217b35e2-7651-45dc-993f-e5c3e25174b3 >2014-10-23 14:32:27,375 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-97) [74c3027a] CommandMultiAsyncTasks::AttachTask: Attaching task 8f662aa5-9d07-4f4c-b8fc-0f2987b55ff2 to command 217b35e2-7651-45dc-993f-e5c3e25174b3. >2014-10-23 14:32:27,383 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-97) [74c3027a] Adding task 8f662aa5-9d07-4f4c-b8fc-0f2987b55ff2 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-10-23 14:32:27,395 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-97) [74c3027a] Lock freed to object EngineLock [exclusiveLocks= key: 1742b7af-bb81-4e6b-bf10-feab68a2f799 value: DISK >key: a1341057-9cf8-4750-a734-656d05a5d2c5 value: STORAGE >key: dfba228f-46f3-4758-a4f0-05857683a088 value: DISK >, sharedLocks= key: f8f5eaee-8fd0-4b45-87db-62d61b03a916 value: OVF_UPDATE >] >2014-10-23 14:32:27,400 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-97) [74c3027a] START, SetVolumeDescriptionVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = dfba228f-46f3-4758-a4f0-05857683a088, imageId = 26155adf-9478-4927-8c49-8e12088c8648), log id: 45accb18 >2014-10-23 14:32:27,401 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-97) [74c3027a] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-10-23 14:32:27,401 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-97) [74c3027a] -- setVolumeDescription parameters: > spUUID=f8f5eaee-8fd0-4b45-87db-62d61b03a916 > sdUUID=a1341057-9cf8-4750-a734-656d05a5d2c5 > imageGroupGUID=dfba228f-46f3-4758-a4f0-05857683a088 > volUUID=26155adf-9478-4927-8c49-8e12088c8648 > description={"Updated":true,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"a1341057-9cf8-4750-a734-656d05a5d2c5"}],"Last Updated":"Thu Oct 23 14:32:25 IDT 2014","Size":71680} > >2014-10-23 14:32:27,555 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-97) [74c3027a] FINISH, SetVolumeDescriptionVDSCommand, log id: 45accb18 >2014-10-23 14:32:27,569 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-97) [74c3027a] START, SetVolumeDescriptionVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = 1742b7af-bb81-4e6b-bf10-feab68a2f799, imageId = 5b2b66c8-425d-4c49-9b3c-62f7586aac1b), log id: 2fcb4aed >2014-10-23 14:32:27,569 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-97) [74c3027a] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-10-23 14:32:27,569 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-97) [74c3027a] -- setVolumeDescription parameters: > spUUID=f8f5eaee-8fd0-4b45-87db-62d61b03a916 > sdUUID=a1341057-9cf8-4750-a734-656d05a5d2c5 > imageGroupGUID=1742b7af-bb81-4e6b-bf10-feab68a2f799 > volUUID=5b2b66c8-425d-4c49-9b3c-62f7586aac1b > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"a1341057-9cf8-4750-a734-656d05a5d2c5"}],"Last Updated":"Wed Oct 22 17:32:20 IDT 2014"} > >2014-10-23 14:32:27,650 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-97) [74c3027a] FINISH, SetVolumeDescriptionVDSCommand, log id: 2fcb4aed >2014-10-23 14:32:27,671 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-97) [1df748c2] Running command: UploadStreamCommand internal: true. Entities affected : ID: a1341057-9cf8-4750-a734-656d05a5d2c5 Type: Storage >2014-10-23 14:32:27,673 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-97) [1df748c2] START, UploadStreamVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6), log id: 70d42616 >2014-10-23 14:32:27,673 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-97) [1df748c2] -- executeVdsBrokerCommand: >2014-10-23 14:32:27,673 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-97) [1df748c2] -- parameters: > spUUID=f8f5eaee-8fd0-4b45-87db-62d61b03a916 > sdUUID=a1341057-9cf8-4750-a734-656d05a5d2c5 > imageGUID=1742b7af-bb81-4e6b-bf10-feab68a2f799 > volUUID=5b2b66c8-425d-4c49-9b3c-62f7586aac1b > size=71,680 > >2014-10-23 14:32:29,085 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-97) [1df748c2] FINISH, UploadStreamVDSCommand, log id: 70d42616 >2014-10-23 14:32:29,088 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-97) [1df748c2] CommandMultiAsyncTasks::AttachTask: Attaching task 5f734500-bf74-439f-87ac-362aac3d038d to command 217b35e2-7651-45dc-993f-e5c3e25174b3. >2014-10-23 14:32:29,098 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-97) [1df748c2] Adding task 5f734500-bf74-439f-87ac-362aac3d038d (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-10-23 14:32:29,102 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-97) [1df748c2] Trying to release a shared lock for key: f8f5eaee-8fd0-4b45-87db-62d61b03a916OVF_UPDATE , but lock does not exist >2014-10-23 14:32:29,102 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-97) [1df748c2] Trying to release exclusive lock which does not exist, lock key: 1742b7af-bb81-4e6b-bf10-feab68a2f799DISK >2014-10-23 14:32:29,102 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-97) [1df748c2] Trying to release exclusive lock which does not exist, lock key: a1341057-9cf8-4750-a734-656d05a5d2c5STORAGE >2014-10-23 14:32:29,102 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-97) [1df748c2] Trying to release exclusive lock which does not exist, lock key: dfba228f-46f3-4758-a4f0-05857683a088DISK >2014-10-23 14:32:29,102 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-97) [1df748c2] Lock freed to object EngineLock [exclusiveLocks= key: 1742b7af-bb81-4e6b-bf10-feab68a2f799 value: DISK >key: a1341057-9cf8-4750-a734-656d05a5d2c5 value: STORAGE >key: dfba228f-46f3-4758-a4f0-05857683a088 value: DISK >, sharedLocks= key: f8f5eaee-8fd0-4b45-87db-62d61b03a916 value: OVF_UPDATE >] >2014-10-23 14:32:29,104 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-97) [1df748c2] START, SetVolumeDescriptionVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, storageDomainId = a1341057-9cf8-4750-a734-656d05a5d2c5, imageGroupId = 1742b7af-bb81-4e6b-bf10-feab68a2f799, imageId = 5b2b66c8-425d-4c49-9b3c-62f7586aac1b), log id: cf27794 >2014-10-23 14:32:29,104 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-97) [1df748c2] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-10-23 14:32:29,104 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-97) [1df748c2] -- setVolumeDescription parameters: > spUUID=f8f5eaee-8fd0-4b45-87db-62d61b03a916 > sdUUID=a1341057-9cf8-4750-a734-656d05a5d2c5 > imageGroupGUID=1742b7af-bb81-4e6b-bf10-feab68a2f799 > volUUID=5b2b66c8-425d-4c49-9b3c-62f7586aac1b > description={"Updated":true,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"a1341057-9cf8-4750-a734-656d05a5d2c5"}],"Last Updated":"Thu Oct 23 14:32:25 IDT 2014","Size":71680} > >2014-10-23 14:32:29,230 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-97) [1df748c2] FINISH, SetVolumeDescriptionVDSCommand, log id: cf27794 >2014-10-23 14:32:29,237 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-97) [1df748c2] BaseAsyncTask::startPollingTask: Starting to poll task 8f662aa5-9d07-4f4c-b8fc-0f2987b55ff2. >2014-10-23 14:32:29,237 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-97) [1df748c2] BaseAsyncTask::startPollingTask: Starting to poll task 5f734500-bf74-439f-87ac-362aac3d038d. >2014-10-23 14:32:35,331 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-31) Polling and updating Async Tasks: 2 tasks, 2 tasks to poll now >2014-10-23 14:32:35,343 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-31) SPMAsyncTask::PollTask: Polling task 8f662aa5-9d07-4f4c-b8fc-0f2987b55ff2 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-10-23 14:32:35,344 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-31) BaseAsyncTask::onTaskEndSuccess: Task 8f662aa5-9d07-4f4c-b8fc-0f2987b55ff2 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-10-23 14:32:35,344 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-31) Task with DB Task ID fcc991ea-125a-4775-8c6b-4766699e894e and VDSM Task ID 5f734500-bf74-439f-87ac-362aac3d038d is in state Polling. End action for command 217b35e2-7651-45dc-993f-e5c3e25174b3 will proceed when all the entitys tasks are completed. >2014-10-23 14:32:35,345 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-31) SPMAsyncTask::PollTask: Polling task 5f734500-bf74-439f-87ac-362aac3d038d (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-10-23 14:32:35,345 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-31) BaseAsyncTask::onTaskEndSuccess: Task 5f734500-bf74-439f-87ac-362aac3d038d (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-10-23 14:32:35,345 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-31) CommandAsyncTask::endActionIfNecessary: All tasks of command 217b35e2-7651-45dc-993f-e5c3e25174b3 has ended -> executing endAction >2014-10-23 14:32:35,345 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-31) CommandAsyncTask::endAction: Ending action for 2 tasks (command ID: 217b35e2-7651-45dc-993f-e5c3e25174b3): calling endAction . >2014-10-23 14:32:35,345 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-49) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction ProcessOvfUpdateForStorageDomain, executionIndex: 0 >2014-10-23 14:32:35,382 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (org.ovirt.thread.pool-7-thread-49) [53492f05] Ending command successfully: org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand >2014-10-23 14:32:35,383 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-49) [53492f05] Trying to release a shared lock for key: f8f5eaee-8fd0-4b45-87db-62d61b03a916OVF_UPDATE , but lock does not exist >2014-10-23 14:32:35,383 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-49) [53492f05] Trying to release exclusive lock which does not exist, lock key: 1742b7af-bb81-4e6b-bf10-feab68a2f799DISK >2014-10-23 14:32:35,383 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-49) [53492f05] Trying to release exclusive lock which does not exist, lock key: a1341057-9cf8-4750-a734-656d05a5d2c5STORAGE >2014-10-23 14:32:35,383 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-7-thread-49) [53492f05] Trying to release exclusive lock which does not exist, lock key: dfba228f-46f3-4758-a4f0-05857683a088DISK >2014-10-23 14:32:35,383 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (org.ovirt.thread.pool-7-thread-49) [53492f05] Lock freed to object EngineLock [exclusiveLocks= key: 1742b7af-bb81-4e6b-bf10-feab68a2f799 value: DISK >key: a1341057-9cf8-4750-a734-656d05a5d2c5 value: STORAGE >key: dfba228f-46f3-4758-a4f0-05857683a088 value: DISK >, sharedLocks= key: f8f5eaee-8fd0-4b45-87db-62d61b03a916 value: OVF_UPDATE >] >2014-10-23 14:32:35,383 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-49) [53492f05] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type ProcessOvfUpdateForStorageDomain completed, handling the result. >2014-10-23 14:32:35,383 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-49) [53492f05] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type ProcessOvfUpdateForStorageDomain succeeded, clearing tasks. >2014-10-23 14:32:35,383 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-49) [53492f05] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 5f734500-bf74-439f-87ac-362aac3d038d >2014-10-23 14:32:35,384 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-49) [53492f05] START, SPMClearTaskVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, taskId = 5f734500-bf74-439f-87ac-362aac3d038d), log id: 70c49637 >2014-10-23 14:32:35,385 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-49) [53492f05] START, HSMClearTaskVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, taskId=5f734500-bf74-439f-87ac-362aac3d038d), log id: 7a622eb7 >2014-10-23 14:32:35,420 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-49) [53492f05] FINISH, HSMClearTaskVDSCommand, log id: 7a622eb7 >2014-10-23 14:32:35,420 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-49) [53492f05] FINISH, SPMClearTaskVDSCommand, log id: 70c49637 >2014-10-23 14:32:35,424 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-49) [53492f05] BaseAsyncTask::removeTaskFromDB: Removed task 5f734500-bf74-439f-87ac-362aac3d038d from DataBase >2014-10-23 14:32:35,424 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-49) [53492f05] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 8f662aa5-9d07-4f4c-b8fc-0f2987b55ff2 >2014-10-23 14:32:35,425 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-49) [53492f05] START, SPMClearTaskVDSCommand( storagePoolId = f8f5eaee-8fd0-4b45-87db-62d61b03a916, ignoreFailoverLimit = false, taskId = 8f662aa5-9d07-4f4c-b8fc-0f2987b55ff2), log id: 6d922c6d >2014-10-23 14:32:35,427 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-49) [53492f05] START, HSMClearTaskVDSCommand(HostName = cyan-vdsg.qa.lab.tlv.redhat.com, HostId = 3a2a45bd-4411-4428-aa45-a514d30d3ea6, taskId=8f662aa5-9d07-4f4c-b8fc-0f2987b55ff2), log id: b76c0a9 >2014-10-23 14:32:35,461 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-49) [53492f05] FINISH, HSMClearTaskVDSCommand, log id: b76c0a9 >2014-10-23 14:32:35,461 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-7-thread-49) [53492f05] FINISH, SPMClearTaskVDSCommand, log id: 6d922c6d >2014-10-23 14:32:35,466 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-7-thread-49) [53492f05] BaseAsyncTask::removeTaskFromDB: Removed task 8f662aa5-9d07-4f4c-b8fc-0f2987b55ff2 from DataBase >2014-10-23 14:32:35,466 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-7-thread-49) [53492f05] CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 217b35e2-7651-45dc-993f-e5c3e25174b3 >2014-10-23 14:33:50,134 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-95) Setting new tasks map. The map contains now 0 tasks >2014-10-23 14:33:50,135 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-95) Cleared all tasks of pool f8f5eaee-8fd0-4b45-87db-62d61b03a916. >2014-10-23 14:35:25,736 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-78) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host alma06.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-23 14:35:40,895 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-22) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host cyan-vdsg.qa.lab.tlv.redhat.com has time-drift of 419 seconds while maximum configured value is 300 seconds. >2014-10-23 14:47:47,117 INFO [org.ovirt.engine.core.bll.ChangeVMClusterCommand] (ajp-/127.0.0.1:8702-8) [47517aea] Running command: ChangeVMClusterCommand internal: false. Entities affected : ID: 3ff759c9-d2a8-4452-8527-f9652ed7f22b Type: VMAction group EDIT_VM_PROPERTIES with role type USER, ID: 67866b36-fd68-4106-8758-34cf31b0c3d4 Type: VdsGroupsAction group CREATE_VM with role type USER >2014-10-23 14:47:47,128 INFO [org.ovirt.engine.core.bll.ChangeVMClusterCommand] (ajp-/127.0.0.1:8702-8) [47517aea] Due to cluster change, removing VM from associated affinity group(s): test >2014-10-23 14:47:47,140 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-8) [47517aea] Correlation ID: 47517aea, Call Stack: null, Custom Event ID: -1, Message: VM affinity_vm-3 configuration was updated by admin. >2014-10-23 14:47:47,261 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp-/127.0.0.1:8702-6) [4345377b] Lock Acquired to object EngineLock [exclusiveLocks= key: affinity_vm-3 value: VM_NAME >, sharedLocks= key: 3ff759c9-d2a8-4452-8527-f9652ed7f22b value: VM >] >2014-10-23 14:47:47,331 WARN [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp-/127.0.0.1:8702-6) [4345377b] CanDoAction of action UpdateVm failed. Reasons:VAR__ACTION__UPDATE,VAR__TYPE__VM,ACTION_TYPE_CPU_PROFILE_NOT_MATCH_CLUSTER >2014-10-23 14:47:47,332 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp-/127.0.0.1:8702-6) [4345377b] Lock freed to object EngineLock [exclusiveLocks= key: affinity_vm-3 value: VM_NAME >, sharedLocks= key: 3ff759c9-d2a8-4452-8527-f9652ed7f22b value: VM >] >2014-10-23 14:47:57,207 INFO [org.ovirt.engine.core.bll.ChangeVMClusterCommand] (ajp-/127.0.0.1:8702-5) [60d18208] Running command: ChangeVMClusterCommand internal: false. Entities affected : ID: 3ff759c9-d2a8-4452-8527-f9652ed7f22b Type: VMAction group EDIT_VM_PROPERTIES with role type USER, ID: 08ea0169-ae42-4e1b-bb20-839a66cd2010 Type: VdsGroupsAction group CREATE_VM with role type USER >2014-10-23 14:47:57,222 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [60d18208] Correlation ID: 60d18208, Call Stack: null, Custom Event ID: -1, Message: VM affinity_vm-3 configuration was updated by admin. >2014-10-23 14:47:57,304 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp-/127.0.0.1:8702-7) [16b8227c] Lock Acquired to object EngineLock [exclusiveLocks= key: affinity_vm-3 value: VM_NAME >, sharedLocks= key: 3ff759c9-d2a8-4452-8527-f9652ed7f22b value: VM >] >2014-10-23 14:47:57,347 WARN [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp-/127.0.0.1:8702-7) [16b8227c] CanDoAction of action UpdateVm failed. Reasons:VAR__ACTION__UPDATE,VAR__TYPE__VM,ACTION_TYPE_CPU_PROFILE_NOT_MATCH_CLUSTER >2014-10-23 14:47:57,347 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp-/127.0.0.1:8702-7) [16b8227c] Lock freed to object EngineLock [exclusiveLocks= key: affinity_vm-3 value: VM_NAME >, sharedLocks= key: 3ff759c9-d2a8-4452-8527-f9652ed7f22b value: VM >]
<b>You cannot view the attachment while viewing its details because your browser does not support IFRAMEs. <a href="attachment.cgi?id=949839">View the attachment on a separate page</a>.</b>
View Attachment As Raw
Actions:
View
Attachments on
bug 1156018
: 949839 |
949840